Skip to content
Hochschule für Informatik FHNW P9 — Mars Rover Autonomy

Demo · runs in your browser

SmolVLA + RECAP + guidance, client-side

The policy family from contribution 01, solving PushT with no backend at all: Chipmunk2D physics compiled to WASM, policy inference on WebGPU through onnxruntime-web. Watch it push the T onto the goal, drag the agent away and let the policy recover, and move the guidance weight to see what it changes.

What you are looking at

A PD-controlled agent pushes a rigid T onto a target pose. The policy sees only a 96×96 image and its own position, and its action is a target position — sixteen of them per chunk. That last detail is what makes the demo legible: because an action is a position, a sampled action chunk can be drawn straight onto the arena as a trajectory. The fan you see is sixteen independent draws from the flow-matching policy, all conditioned on one frozen observation.

Dragging inside the arena is therefore a valid action, not a special case — take over mid-episode, put the agent somewhere awkward, release, and watch the policy resume from a state it never saw in training.

Worth being precise about

Raising the guidance weight measurably changes what the policy achieves. It does not visibly narrow the fan — the spread of sampled headings is the same at high guidance as at zero. The intuition that guidance collapses a distribution comes from language-conditioned image models and does not carry over to a single advantage token.

The measured numbers are printed next to the demo itself, generated from the same evaluation run that produced its starting states rather than copied by hand.

The demo is embedded on this page once it is deployed. It stays a separate build on its own URL so it can be opened standalone, and so a page of thesis prose never has to ship alongside half a gigabyte of model weights.