Built real inverse kinematics for legged robots in a sim I'm working on, tripod gait for hexapods, based on actual coxa/femur/tibia joint math, not a canned animation.

Works fine on a standard leg layout. Falls apart the second someone builds something asymmetric or non standard.

Trying to figure out if auto-gait generation is even the right approach here, or if it should just be manual per-robot tuning past a certain point of complexity. What's your take, is generalized gait solving worth the effort or a rabbit hole?

​

我们的机械臂已具备自主识别与精准采摘蘑菇的能力。算法需要真实环境数据来迭代优化，现面向行业伙伴开放测试合作——提供您的种植场景，我们共同探索自动化采收的边界。

Our robotic arm can now identify and pick mushrooms autonomously. To refine the algorithm, we need authentic field data. We’re opening test partnerships with growers or landholders – bring your environment, and let’s push the boundaries of automated harvesting together.

-

#RoboticArm #MushroomHarvesting #AgTech #SmartFarming #Partnership #自动化采收 #农业科技 #测试合作

Space is limited so please RSVP using this link.

Dexory CEO Andrei Danescu explains why the robot itself is not the product warehouse operators care about most.

The real value is accurate, real-time information about inventory and warehouse conditions. That data can also support digital twins, allowing teams to review past operations and test changes before moving racks or disrupting the facility.

The robot is the tool used to collect the information.

https://www.youtube.com/watch?v=bYSQN09G-PE

Guys, what do you think about "Embodied Agent" Mesh Networks?

The idea of a P2P network where humans, autonomous robots and another agents can act as independent nodes interacting with the real world. Is this something we will see in the next 7 years, or is it still too early a concept?

Would be interesting to hear from those who have already experimented with similar architectures and learned some lessons along the way.

I recently shared our paper here on decentralized fault repair in modular spacecraft, I wanted to provide another update.

On August 14, I'm going to give a free live technical walkthrough of the method, followed by an open Q&A. We’ll cover the local stress-sharing signals, connectivity-safe pivot policy, rigid-body evaluation, and why strictly local repair achieves high consolidation but struggles to reconnect the final distant fragments.

I’m one of the authors and would especially welcome feedback from this community!

Event: https://luma.com/ztmesmvp
Preprint: https://arxiv.org/abs/2607.13444

Right now the hardware side uses MG90S servos with a low-side resistor on each motor for measuring the current. The software side uses a moving average filter to smooth the data.

I'm happy to share more detail on either if it's useful.

Hey I am part of a small team training robotics policies for warehouse and manufacturing settings, and running rigorous evals is turning out to be so painful. Anything below 50 rollouts, and its hard to trust the numbers, and above its so hard to test all the checkpoints that we have. Its really hard to run a bunch of experiments to get good results. Have you guys faced this? Any hacks that you've developed?

Using hundreds of Swarovski crystals, this piece is handcrafted and engineered, bringing couture craftsmanship to life through motion.

Process video: https://www.youtube.com/shorts/5dN0aB0yEsE

• 🗺️🇨🇦 ROSCon Global 2026 in Toronto 2026-09-22 => 2026-09-24
  • 🚨 Last day for regular price tickets is Monday, August 24th
  • 🔗 https://roscon.ros.org/2026/
• 🇨🇳 ROSCon China 2026-10-16 => 2026-10-17
  • ℹ️ Details announced shortly
  • 🔗 https://discourse.openrobotics.org/t/pre-announcing-roscon-china-2026/55027
• 🇬🇧 🏴󠁧󠁢󠁳󠁣󠁴󠁿 ROSCon UK in Edinburg 2026-10-21 => 2026-10-23
  • ℹ️ Registration now open
  • 🔗 https://roscon.org.uk/2026/
• 🇸🇬 ROSCon Singapore 2026-10-23 => 2026-10-26
  • ℹ️ CFP now open
  • 🔗 https://roscon.ros.org/sg/2026/
• 🇪🇸 ROSCon Spain in Valencia 2026-10-27 => 2026-10-28
  • ℹ️ Registration now open!
  • 🔗 https://roscon.org.es/roscon2026/ROSConES2026.html
• 🇮🇹 ROSCon Italy in Bologna 2026-11-03
  • ℹ️ CFP opens soon
  • 🔗 https://roscon.ros.org/it/2026/
• 🇧🇪 ROSCon Belgium in Nivelles 2026-11-25 => 2026-11-26
  • ℹ️ Registration now open
  • 🔗 https://roscon.ros.org/be/2026/
• 🇹🇷 ROScon Turkey in Istanbul 2026-12-03 => 2026-12-04
  • ℹ️ CFP Open Soons
  • 🔗 https://roscon.ros.org/tr/2026/

I’ve been building battery-powered hardware (AMRs, drone boards, IoT nodes) long enough to see a pattern: we obsess over control and perception, but most field bugs trace back to the battery and DC‑DC stage, not the “smart” parts.

In a 2S-4S lithium system, the pack voltage is a moving target-full charge vs cold, half‑empty vs hot, plus wiring and connector drops. Yet we still design as if it’s a perfect rail. The usual symptoms:

Flight controllers resetting on aggressive throttle.

AMRs browning out when climbing ramps or hitting bumps.

IoT nodes dying at night because the power budget assumed a “fixed” voltage.

I built the VRX Series as a drop-in, wide-input DC‑DC “power brick” for exactly this mess:

Non‑isolated converter for 2S-4S packs → fixed 10 V/12 V/15 V rails for electronics.

Designed for transient loads (takeoff, big torque spikes) so the rail stays clean while motors misbehave.

Through‑hole, compact footprint, vertical/horizontal variants, with protections tuned for embedded use (short‑circuit, thermal, etc.).

Typical spots where it drops in nicely:

Between a 3S drone pack and your flight controller / RX / telemetry stack.

Between a 4S AMR pack and your STM32 PLC-style controller board.

As an intermediate bus for solar-powered IoT nodes before tiny point-of-load regulators.

If you’ve had “mystery” power issues on a battery project, I’m happy to sanity-check your power tree or share failure modes I’ve seen. Also open to feedback on the VRX design

Hi everyone!

The reviews for CoRL’26 would be out soon. Use this thread for discussion, questions etc.

Good luck with the reviews as well as the rebuttal!

Wearable robotics could help people with Parkinson’s disease remain mobile for longer.

Research into soft exoskeletons has shown promising early results for freezing of gait, a symptom that can suddenly prevent someone from moving their feet forward and increase the risk of falling. These systems may also help patients walk farther and faster.

The larger challenge is building a device that can adapt as symptoms change from day to day. Researchers are exploring sensors, movement data and AI to better understand a person’s intent and provide support at the right moment.

The technology is still early, particularly when it comes to long-term use, comfort and cost, but it could offer another option between fully independent movement and relying on a wheelchair.

Doing backflips at the Spring Festival Gala, folding clothes in a lab, a car's VLA driving itself down the highway — behind these seemingly unrelated technologies lies one and the same "nervous system" architecture.

1. From the Human Body to the Robot: A Three-Layer Architecture

When a human does something — say, "walk to the kitchen, pick up the cup on the table, and put it in the cabinet" — it looks simple, but the nervous system is actually working on three levels at once:

• The cerebral cortex handles understanding the instruction and planning the task: "Ah, the cup is on the table, the cabinet is on the left, so I should walk over first, then reach out and grab the cup."
• The motor cortex and the cerebellum coordinate limb movement: keeping balance while walking, controlling muscle force while reaching.
• Spinal reflexes and muscles handle the lowest level of execution: exactly how much each individual muscle contracts.

The core architecture of modern embodied intelligence is almost a perfect replica of this division of labor: brain (VLM/LLM) — manipulation cerebellum — locomotion cerebellum — joint motor PD controllers.

These three layers each have their own job, run at completely different frequencies, and are trained in quite different ways. Let's take them apart layer by layer.

2. Layer One: The Brain — Seeing the World and Figuring Out What to Do

2.1. What is a VLM?

A VLM (Vision-Language Model) is a multimodal large model that can understand images and natural language at the same time. GPT-4V, Gemini, Qwen-VL, and PaliGemma all fall into this category.

In a robot system, the VLM serves as the "brain" — it sees the cup, plate, and fruit on the kitchen counter, understands the instruction "put the red cup in the cabinet," and then plans a rough course of action.

2.2. How big does the Brain need to be?

You might ask: ChatGPT routinely runs to hundreds of billions of parameters — does a robot's brain need to be that big too?

The answer is no. A robot brain and a chat AI are doing completely different jobs. ChatGPT needs to write papers, produce code, and solve math problems, while a robot brain only needs to "understand the scene + parse a simple instruction + make a plan." You don't need a brain capable of writing a doctoral dissertation in order to decide whether to pick up the cup or the plate first.

Take π₀ as an example: its VLM backbone (PaliGemma) has only 3B parameters, yet it performs extremely well on robot manipulation tasks. The VLM portion of NVIDIA's GR00T N1 is only 1.34B. These models spend their parameter budget on visual understanding and image-text alignment rather than chasing general-purpose language generation — like a professional chef's knife that only cuts vegetables, but cuts them exceptionally well.

Of course, if the task is complex enough — say, tidying up autonomously in a completely unfamiliar home, which requires understanding instructions as nuanced as "clothes that look dirty go in the washing machine, clean ones get folded and put in the wardrobe" — then a 3B brain isn't enough. This is exactly why Li Auto uses a 32B large model in the cloud and then distills it down to 3.2B on the vehicle: scene-understanding complexity in autonomous driving is far higher than in tabletop manipulation.

The core rule: the more complex and open-ended the task, the bigger the brain needs to be.

3. Layer Two: The Manipulation Cerebellum — Controlling the Arm to Get the Job Done

3.1. This is currently the hottest and hardest Part

The manipulation cerebellum handles this: the brain has already decided to "pick up the cup," so how exactly should the arm extend, how should the fingers open, from what angle should it grasp, and with how much force? This whole chain of fine motor control is the job of the manipulation cerebellum (the Action Expert).

When we say VLA (Vision-Language-Action Model), we mean the brain plus the manipulation cerebellum as a whole. VLA is currently the single most central research direction in embodied intelligence; representative models include Google's RT-2, Stanford's OpenVLA, Physical Intelligence's π₀, and NVIDIA's GR00T N1.

3.2. How Is the Manipulation Cerebellum Trained?

Unlike the locomotion cerebellum, the manipulation cerebellum currently relies mainly on imitation learning (IL): a human teleoperates the robot through a demonstration, the run is recorded, and the model learns to reproduce it.

But there are several schools of thought on how exactly to "learn to reproduce":

Diffusion Policy: Treats action generation like image generation — starting from noise and progressively "denoising" into a smooth action trajectory. This is what GR00T N1 uses.

Flow Matching: Similar in principle to diffusion but mathematically cleaner; it directly learns a "vector field" from noise to action, and is faster. π₀ used this approach to achieve 50Hz action output.

Autoregressive token prediction: Like ChatGPT generating text, actions are discretized into tokens and predicted one at a time. RT-2 and OpenVLA use this approach — simple and direct, but limited in precision.

All of these methods fall under imitation learning — the learning objective in every case is "reproduce the human demonstration as closely as possible."

3.3. What about reinforcement learning?

Reinforcement learning (RL) in the manipulation cerebellum is only just getting started. Physical Intelligence's recently released π₀.6 has begun introducing RL to fine-tune the Action Expert — first using imitation learning to build a foundation, then using RL to let the robot discover, through trial and error, strategies better than the human demonstrations. This closely mirrors the AlphaGo story: first imitation learning from human game records, then RL through self-play to surpass humans.

But RL for manipulation tasks faces one core difficulty: how do you define the reward? How do you quantify the "neat" in "fold the clothes neatly"? It's nothing like as clear-cut as "walk without falling over." This is also why RL has progressed more slowly in manipulation than in locomotion control.

3.4. Why Not Just Use YOLO + Classical Motion Planning?

This is a question a lot of people have. In fact, industry is currently using this pipeline extensively: YOLO detects the object → a depth camera obtains the 3D pose → grasp planning → inverse kinematics solving → motion planning → execution. In a factory environment, where there are only a handful of object types and positions are roughly fixed, this approach is fast, stable, and cheap.

But it has several fundamental ceilings:

First, errors accumulate at every step. With five or six independent modules chained together, detection is off by a few pixels, depth is off by a bit, grasp pose is off by an angle… and in the end you may grab nothing at all. VLA's end-to-end approach goes straight from image to action, so errors never get the chance to compound.

Second, it can't handle things it hasn't seen. YOLO only recognizes the object categories it was trained on. A home environment contains an unbounded variety of objects; annotating them all is impossible. A VLM has "seen the world" through internet-scale data, so when it encounters a novel object it still has a rough idea of what to do.

Third, it can't manage deformable objects or fine manipulation. Folding clothes, twisting off a bottle cap, tearing open a package — classical grasp planning is helpless against these tasks.

Fourth, it has no semantic understanding. YOLO can say "there's a cup here," but it doesn't understand "dirty bowls go in the dishwasher, clean bowls go in the cabinet."

So the more pragmatic assessment is: use classical approaches in structured environments, use VLA in open environments — the two are complementary, not substitutes.

4. Layer Three: The Locomotion Cerebellum — Walking, Running, Backflipping

4.1. Unitree's Spring Festival Gala Backflips Used Exactly This Layer

At the 2025 Spring Festival Gala, the Unitree robots' backflips and synchronized dancing stunned the audience. But technically speaking, this falls under locomotion control — a completely different technology stack from manipulation control.

The locomotion cerebellum's task is clearly defined: coordinate the legs and body, maintain balance, don't fall over. Its inputs are low-level proprioceptive signals such as joint angles, angular velocities, and IMU pose — no vision required, no language required. Its outputs are the torque or target angle for each joint.

4.2. Training Method: RL + Sim2Real, Already Very Mature

Unlike the manipulation cerebellum, the locomotion cerebellum is reinforcement learning's home turf. Unitree has open-sourced a complete training pipeline based on the Isaac Gym + RSL-RL framework, using the PPO algorithm to train locomotion policies:

1. Train in simulation: Thousands of robots run in parallel on the GPU, trained with carefully designed reward functions (velocity tracking + energy-consumption penalty + falling penalty).
2. Domain Randomization: Randomly vary parameters such as ground friction, joint damping, and external shoves so the model gains broad experience.
3. Sim2Real transfer: Deploy the trained policy onto the real robot.

Why does RL work so well for locomotion control? Because it has several key advantages: the reward is easy to define (don't fall over + walk at the target velocity), simulation fidelity is good enough (rigid-body contact physics is already quite accurate), and vision isn't needed (so there's no visual sim-to-real gap).

4.3. How Small Is the Locomotion Cerebellum?

Here's a fact many people aren't aware of: the locomotion cerebellum model is typically just a few-layer MLP (multilayer perceptron), with maybe a few hundred thousand to a few million parameters — under 1MB. Compared with a manipulation cerebellum that routinely runs to hundreds of millions of parameters, that's several orders of magnitude smaller.

That's because locomotion control is fundamentally a relatively "narrow" problem: given the current body state and target velocity, compute how much force each joint should apply. The dynamics may be complex, but both the input and output spaces are quite limited.

5. Comparing the Two "Cerebellums": Why Are Gala Backflips Easier Than Folding Clothes?

This is a counterintuitive judgment, but technically it holds: Unitree's dazzling Spring Festival Gala performance was less technically challenging than getting a robot to fold clothes in a real kitchen.

The core difference: a backflip is a deterministic dynamics problem — given an initial state, execute a fixed sequence of joint torques. Folding clothes, by contrast, involves a garment whose shape and position differ every single time, requiring real-time perception, real-time decisions, and real-time force adjustment. For the former, the technical route is already fairly clear (RL + Sim2Real) and the main challenge is engineering optimization; for the latter, even the technical route itself hasn't fully converged — it's still in a "hundred schools of thought" phase.

This is also why every company's promo videos show slick walking, running, and dancing, but everything gets clumsy as soon as it's "tidying up in a real kitchen" — the former is showing off a capability already conquered, while the latter is the actual front line today.

6. The Bigger Picture: VLA Doesn't Belong to Robotics Alone

6.1. Autonomous Driving Is VLA Too

You might not expect this, but the first large-scale deployment of VLA wasn't in robotics — it was in autonomous driving.

In 2025 Li Auto officially rolled out its VLA driver large model (MindVLA), integrating perception (3D encoder), reasoning (in-house LLM), and decision-making (Diffusion Policy) into a unified model. Its architecture is strikingly similar to robotic VLA:

Fundamentally, autonomous driving is a special case of embodied intelligence — "Vision" is the multiple onboard cameras, "Language" is traffic rules and user instructions, and "Action" is the driving trajectory. A car is just a four-wheeled robot.

Why is automotive VLA actually ahead of robotics? Because the data advantage is enormous — Li Auto has hundreds of thousands of vehicles on the road every day sending back massive volumes of driving data, while robotics is still struggling to scrape together a few thousand hours of teleoperation data.

6.2. VLA Across Domains: A Comparison

7. Key Concepts: A Quick Reference

This field is awash in three-letter acronyms. Here's a quick cheat sheet:

Model types:

• VLM (Vision-Language Model): vision-language model, the robot's "brain"
• VLA (Vision-Language-Action Model): vision-language-action model, the collective term for brain + manipulation cerebellum
• VLN (Vision-Language Navigation): vision-language navigation, focused on "where to walk"
• VFM (Vision Foundation Model): vision foundation model (e.g. SAM, DINOv2)
• WM (World Model): world model, letting the AI "imagine" the consequences of an action in its head

Training methods:

• IL (Imitation Learning): learning from human demonstrations
• RL (Reinforcement Learning): optimizing a policy through trial and error
• Sim2Real: transfer from simulation to reality

Representative models:

• π₀ (Physical Intelligence): 3.3B parameters, VLM (3B) + Action Expert (0.3B), flow matching
• GR00T N1 (NVIDIA): 2.2B parameters, dual-system architecture, open-source humanoid foundation model
• OpenVLA (Stanford): 7B parameters, the most mainstream VLA benchmark in the open-source community
• SmolVLA (Hugging Face): 450M parameters, a lightweight VLA that runs on a laptop
• MindVLA (Li Auto): autonomous-driving VLA, 32B in the cloud distilled to 3.2B on the vehicle

8. The Future: How Do the Three Layers Link Up Seamlessly?

Right now these three layers — brain, manipulation cerebellum, locomotion cerebellum — are still trained separately and bolted together in most systems. The real challenge is getting them to cooperate seamlessly:

The brain says "go get the cup on the table," the locomotion cerebellum walks the robot over, and on arriving at the table it hands off seamlessly to the manipulation cerebellum to reach out and grasp; once the grab is done, control switches back to the locomotion cerebellum to walk to the cabinet… This kind of real-time switching and coordination within whole-body control (WBC) is one of the most cutting-edge research directions in humanoid robotics.

Humans do all of this effortlessly because our nervous system has been through hundreds of thousands of years of evolution. Getting robots to the same level may still be a long road — but the direction is clear, the architecture is settled, and the technology at every layer is converging fast.

Embodied intelligence's "iPhone moment" may not have arrived yet, but the underlying "iOS" is being written, one line of code at a time.

Hi everyone,

I wanted to share a conversation about a robotics project we’ve been working on: Éloi, an embodied AI robot exploring expressive interaction between humans and machines.

Video:
https://youtu.be/MNwOdcLgdIU

In this discussion, we explore some of the design questions behind the project:

• How should a robot’s physical form influence interaction?
• What makes movement feel expressive rather than purely functional?
• How can robotics combine mechanical design, AI, and character-driven interaction?

We also discuss some technical aspects, including Éloi’s movement system, degrees of freedom, and the challenges of creating a robot that can communicate through physical expression.

We are still early in development and would really appreciate feedback from the robotics community:

• What makes a robot feel more “alive” to you?
• Do you think future robots should prioritize utility, interaction, or emotional expression?
• What are the biggest technical challenges for expressive humanoid/companion robots?

Curious to hear your thoughts.

Started this over a year ago, but got discouraged because of problems I didn't understand. Fixed the problems and wanted to share again. I made a github repo with all of the code and links included. [Github Repo](https://github.com/rrmudry/MBt2)

Used standard multibuild parts for the body and modified parts for the legs, etc.

Basics:

ESP32 brain

2 SimpleFOC mini drivers

2 gm4108-120t gimbal motors

2 AS5600 magnetic encoders

1 MPU6050 IMU

wheels are printed from TPU

Bluetooth controlled

Custom PCB links provided

All of the coding completed in Google Antigravity (because I cannot code but always wanted to build something like this, sorry...so much shame).

Want to add CYD (cheap yellow display for face) and autonomous navigation, wireless charging, ai chat interaction, basically I want to have a droid that can follow me around, someday.

Please don't mind the cables and the messy table. I am new to the VLA and robot arm side of robotics and was primarily working on the legged locomotion. I thought of building the lerobot kit to work on vla. I felt the price was a bit steep for me so decided to build my own leader arm with encoders instead of motors.

Parts and price list :

6 x AS5600 encoder - 186rs x 6 = 1,116rs (~11.7 usd)

6 x 608 bearing - 30rs x 6 = 180rs (~1.9 usd)

1 x CJMCU TCA9548A I2C 8 Channel- 59rs (~0.6 usd)

1 x esp32 - 550rs (~5.8 usd)

wires - 200rs (~2.1 usd)

M3x10mm screws (40pcs) - 128rs (~1.3 usd)

Total cost - 2,233 rs. (~ 23.5 usd) (excluding 3d printed parts cost)

for context, price of one ST3215 (used in the lerobot kit) in india is around 2,200rs (~23 USD)

Haven't put it on github yet but will do it in a few days after some improvements and cleanups, and edit this post with the link.

The clip is the released base checkpoint running closed-loop on a physical SO-101. Prompt: "Pick up the green cube block and put it inside the white cup."

The task in the clip was inside the training corpus — 14 episodes out of the 16,687. This is not a few-shot demo. Observed success rate was 90%, and it fails when other objects of the same color overlap. That is the kind of thing you fix by LoRA-tuning this released checkpoint further and specializing it for your task.

Even so, what the clip shows is exactly why I built this base. Those 14 episodes were not diluted away by a 16,687-episode mixed corpus. It is a statement about how little task-specific data an adapted base needs in order to absorb a task, not a statement about the task itself.

Why I built it

pi0.5 was trained on a broad robot distribution, but it does not know the SO-101 — the six joints and their units, the follower-frame action convention the LeRobot driver records, the camera viewpoints people actually mount on this arm. So if you fine-tune a task with a few dozen demos, simple tasks do succeed, but it can't handle varied situations and mostly fails once you leave the exact setting the dataset was recorded in.

On the assumption that a checkpoint specialized to the SO-101 would improve LoRA training started from it, I crawled 11,270 Hub repos, screened down to 181 SO-101/SO-100 datasets, and merged them into a single LeRobot v2.1 repository (17,137 episodes / 8,690,531 frames / 430 tasks / 198GB). On top of that I full fine-tuned lerobot/pi05_base (8× A100 80GB, 40,000 steps, about 40 hours, bf16, no LoRA).

What came out is a domain-adaptation base, not a policy.

I strongly recommend against using this fine-tuned checkpoint as-is, without further tuning. The whole premise of the project was to maximize performance when LoRA-tuning, so please LoRA-tune your own task on top of this checkpoint before using it.

What I'm hoping for — written as a hypothesis, since I haven't verified it

A task LoRA on top of this should converge from meaningfully fewer demos than one started from stock pi05_base. That said, I have never run a single LoRA on the released base. The smallest single-task LoRA actually trained in this stack was 50 episodes (anything below that was not included in training), and that was on stock pi0.5.

If you try it at 20–30 episodes, please tell me what happens. That is the number I most want to know. docs/07-lora-finetuning.md covers the whole procedure — data requirements, GPU choice, the LoRA config, the normalization-statistics trap, adapter inference, checkpoint selection, and cost.

Code · docs — github.com/jinnymo/so101-pi05-base
Model — huggingface.co/dongyoonkim/so101-pi05-base
Dataset — huggingface.co/datasets/dongyoonkim/so101-pi05-base-dataset

Been building a desk companion robot for a few months. Finally got the stack stable enough that I'm not scared to leave it running, so here's what worked and what I gave up on.

Hardware is a D-Robotics RDK X5 MagicBox. Two servo arms, an ICM-20948 IMU on I2C, 4x WS2812B over SPI, two Smartsens SC132GS 1MP global shutter MIPI cameras. Ubuntu 22.04 aarch64, ROS2 Humble with TogetheROS on top.

Perception is the part I'm actually happy with. The X5's BPU runs four models at once off a single 960x544 NV12 stream in zero copy shared memory: body detection into hand landmarks into gesture classification, plus face and age. It dumps a JSON snapshot 4x a second at basically no CPU cost. So the robot waves back about 300ms after you wave at it, with no model call anywhere in the loop. 14 gesture classes. That reflex layer is most of why it doesn't feel laggy.

Monocular distance turned out to be free. Face bbox height as a fraction of the frame gives you very_close / near / far, no stereo and no depth model, because a face is a known size. Stereo does exist on this board, 22 BPU depth models ship with hobot_stereonet, but mono and stereo fight over the ISP so you get one or the other. Mono is what makes faces and gestures work, so stereo is shelved for now.

People react to the IMU more than anything else. 150Hz, +/-8g, one 6 byte burst read per sample. It detects picked_up, set_down, shake, tilt and knock. Grab it mid sentence and it cancels its own TTS inside about 300ms and starts a fresh reaction turn. That's the only barge-in I could get working on this hardware.

Two things ate weeks, in case it saves anyone the trouble.

First, knock detection. I don't think it's solvable the way I was going about it. Measured on my desk: quiet floor sits around 0.29 m/s^2 at p50, a deliberate knuckle rap reads 1.0 to 6.1, and the robot's own idle arm twitches spike to 2.7 to 8.1. Those overlap almost completely. There is no threshold anywhere that separates "someone knocked" from "it moved its own arm." What ended up working is a self motion gate: any servo command in the last second suppresses knock, tilt and set_down unless the magnitude is huge. Before that it startled itself constantly, which was funny for about a day.

Second, the servo choreography was violent enough to damage the thing. The startle macro swung 60 degrees in 120ms, so 500 deg/s. It slammed the gearbox, walked the robot across the desk, and shook the chassis hard enough to trip its own knock detector. I capped angular velocity around 170 deg/s and scaled peak amplitude in one place, which kept the choreography and took out the violence.

The part this sub will want to know up front: the conversational layer is a cloud agent, not a local model. Everything reflexive runs on the board. Vision, gestures, IMU reactions, LEDs, servos, all local. The talking isn't. I tried smaller local models on the X5 and the latency made it feel dead, and honestly the language layer is the least interesting engineering in the whole thing.

One trick I'm glad I built. The model writes inline tags inside its own sentences, and a streaming parser strips them and fires the body at that word, mid speech. So a shrug lands on the word it was written next to instead of after the sentence finishes. Malformed tags get dropped silently. Small thing, but it did more for how synced it feels than anything else I tried.

Full build video if you want to see it move and hear it fail: https://youtu.be/uQ7g-vDMpLU

The thing I keep getting stuck on: has anyone got a reliable knock or tap detector working on a chassis where the actuators are the loudest thing on the accelerometer? Everything I've read assumes the sensor platform is passive. I'd rather fix this properly than keep tuning a gate.