Fine-tune

Train on your demos

~3 min read

Your dataset is ready. Now it becomes a model.

Fine-tuning turns your demonstrations into a policy the arm can run. It also asks the least of you of any step: one command, a proven recipe, and a curve you can watch to know it's working.

Your demonstrations do the teaching

Your run warm-starts from the pretrained SO-101 base — a model that already knows how to move this arm. Your demonstrations teach it one new skill: the task you recorded, keyed to the sentence you typed. That head start is why a couple dozen good episodes is enough.

The training recipe — step budget, batch size, learning rate — ships proven and tuned for this base. Your model's quality comes from your demonstrations: clean, consistent reps in, capable model out. Bring the data; the recipe does the rest.

The put the red cube in the bowl run is the worked example. It warm-started from the SO-101 base, adapted the action expert only, and trained for ten thousand steps, saving a checkpoint every two thousand — about four hours of GPU time. Nobody picked those numbers; the recipe did.

1 · Launching and watching the run

That one command is pixi run finetune. Pass it the dataset name you chose at record time — nothing else.

pixi run finetune -- --dataset <name>

It exports your episodes, then launches the run — a job handle and watch-page URL print at launch, and the command polls until the run reaches a terminal state. Add --dry-run to stop after the export and print the launch command instead of running it.

Closing the terminal never loses the run: the handle is printed at launch. Re-attach to a running job, or check its state once without watching, using that handle.

newt finetune --handle <job> --status

The run ends in one of two states, and the command tells you which. On success it prints your model's tag and the report card. On failure it names the pipeline gate that stopped the run, rather than a generic error, so you know what to fix instead of guessing. A run never reports success it didn't reach.

You should see

a job handle, a watch-page link, and a heartbeat while the run trains; on success, your model's tag and the exact line to drive it from Python.

A curve you can watch

The watch page shows a live training-loss curve, logged as the run fits your demonstrations and descending as it goes. That descent is the signal that the run is working rather than stalling or diverging. You don't have to interpret anything else to know it's on track.

Recording slot — poster pending

[ learning-curve view — training loss descending across steps ] Placeholder. The loss line falls as the run fits your demonstrations; checkpoints land every 2K steps, through 10K.

It runs while you sleep

On a GPU the run takes a few hours — the red-cube-bowl run was about four — checkpointing as it goes. That's longer than anyone sits and watches, so the event is shaped around it: you record on the first day, the run trains overnight, and you drive the trained model on the second day. Once the curve is descending, the run needs no minding.

When the run succeeds, newt finetune prints your model's tag and the one line that drives it from Python.

from newt import Robot
robot = Robot(model="<your-model-tag>")

You should see

Fine-tune succeeded, your model's tag, and that two-line Python snippet — the model is yours to call by name.

→ Getting started — the installation and platform reference behind newt finetune.

you have a model that learned your task. The next chapter calls it by name.