Why I Want to Push This Work Onto a Physical Robot

One reason I’m increasingly interested in moving grounded language work onto a physical robot is that embodiment changes the meaning of success.

It is one thing for a model to perform well on a dataset. It is another for a system to interpret an instruction, decide whether clarification is needed, and then execute appropriately in the physical world.

That difference matters a lot to me.

A lot of my recent research has been building toward those questions from the data and modeling side through SCOUT++ and the broader Ask-to-Act direction. That work helped clarify the structure of the problem.

The next step I care about is testing more of those ideas in physical interaction rather than leaving them only at the level of datasets and model outputs.

Embodied evaluation forces harder questions. Did the system really understand the instruction? Did it handle uncertainty well? Did it ask at the right time? Did the behavior still make sense once action had real consequences?

To me, that is where grounded language work becomes much more meaningful. Not just as an interpretation task, but as a behavioral problem for intelligent systems operating around people.

---

Originally posted on LinkedIn.