Start with one behaviour
Imagine a robot's localisation component loses the sensor input it needs. Other components use its position estimate to plan and move. What should they receive next?
“Handle missing sensor data” is not specific enough for a user story. The component might keep returning the last position, report an error or stop producing output. Each choice has a different effect on the rest of the robot.
The PO starts at the product level: a robot must not plan from a stale position, and it must recover predictably when the sensor returns. That behaviour becomes a backlog work item. Before asking for code, decide what another component must be able to observe.
Turn the backlog item into a specification people can read
The PO's work item says what the robot needs to do. The engineers turn it into a specification for the current architecture. For localisation, that means identifying the node that owns estimate validity, the interface through which planning receives it, the modules that must change and the modules that must not.
The localisation specification might say: when required input stops arriving, mark the estimate as unavailable within an agreed time. Do not present an old estimate as current and valid. When input returns, recover in an agreed way. It also defines a boundary: localisation says whether its estimate is usable; motion decides how to respond. The engineers agree the interface, limits and error cases with the owners of both components.
Write down these timing requirements, recovery conditions, affected interfaces, expected error handling and what lies outside the change. The result is more detailed than the PO's feature request, because it must be precise enough to evaluate an implementation on this architecture. Here, an agent can help expose gaps; the engineers and PO resolve them.
The PO stays at the higher level: which behaviour matters, which feature combinations belong together and which trade-offs are acceptable. The engineers own architectural quality and implementation checks.
Build the tooling that turns specifications into repeatable work
The Devs/Ops develop tooling, ie the 'harness', that reads the approved specification, prepares the right harness and starts the agents in a controlled environment. For the localisation item, the harness assembles the component, the relevant interfaces, a replay or simulation input and the expected observations. It also records the software branch and exact revision, scenario and configuration so the result can be reproduced.
The acceptance scenario for our localisation example is simple:
- Supply valid sensor input and confirm that the component produces a usable estimate.
- Interrupt that input and check the reported validity and elapsed time against the requirement.
- Restore the input and check the agreed recovery behaviour.
Put this scenario in a protected harness: the environment, inputs and checks used to assess the change. The implementation agent can run it and inspect failures, but it cannot change the acceptance conditions to make its own work pass. If contradictory requirements block progress, engineers resolve the contradiction with the PO where product intent is affected, update the specification and restart the agent loop.
The first useful scenario takes setup. Engineers can use coding assistants to build scenario runners, reports and protection checks, then review that tooling separately from the feature implementation. With an existing build and replay setup, allow weeks to months for this work to mature. A new simulator or hardware rig needs its own schedule. Once built, your setup will serve many changes.
Give agents a protected harness and a repeatable environment
Give the coding agent the architectural specification, relevant interfaces and repeatable environment. It writes code and supporting tests, runs the protected checks and uses the results to improve the change. The tooling collects each run's inputs, outputs and evaluation. Engineers resolve technical blockers; questions that change the requested behaviour go back to the PO.
We don't trust that the agent writes correct tests - they are part of the process but not part of the proof. Agents will create them in order to 'anchor' assumptions of the written code and they will in most cases flag when assumptions are broken by a code change.
One controlled flow, from request to decision
We have now reached phase 3 in the diagram: the implementation runs through the immutable test harness. Phase 4 returns evidence to the agent, closing the inner loop. The agent can improve the code and run the same checks again. The wider human loop uses independent validation to decide whether the result meets the request or whether the specification needs to change.
The agent loops; engineers verify
After a test run, the agent receives evidence: which checks ran, the recorded inputs and outputs, the plots or robot visualisation, and what remains unverified or broken. That evidence drives its next iteration and is available to engineers throughout the loop.
For the localisation example, the evidence report should show the sensor-input trace, the estimate-validity signal, the elapsed time to unavailable and the recovery trace. If the behaviour has a spatial effect, a 3D robot visualisation can show the estimate and the robot's response.
This is where the Devs monitor the agent. Engineers check that the specification was interpreted correctly, that the harness measured the right thing and that the plots make technical sense. They verify what went in and what came out, including the claims in the agent's report. A separate validation agent can compare the change with the original request and evidence, but engineers check its conclusion against the data too.
These views also help engineers check the tooling itself. Did the scenario actually interrupt the sensor input? Does the report show when the estimate became unavailable and when it recovered? Plot the recorded samples against a common time axis, with units and the specified limits visible. Compare the plot with the source data and try a deliberately broken case to check that the evaluator detects it.
And code reviews? Our view is that review effort should shift towards specifications, architecture and the harness. Reading generated code alone gives uneven assurance about robot behaviour. Engineers need to examine whether the checks could expose a wrong implementation, then inspect the observed behaviour. They can still inspect code where the evidence raises questions. The PO adds another level of scrutiny by exercising the resulting features together.
Combine features and visualise behaviour at every level
A robot combines localisation, perception, planning, control, drivers and many other components. You need a graphical interface at every level: the module, the architecture and the full application. Each view answers the same question: what was the input, and what did the software produce, both in isolation and in combination with other components?
For localisation alone, plot sensor arrivals, estimate validity and recovery. Across the architecture, align those signals with the planner's decisions and motion commands. For the full application, show the robot's trajectory and mission progress in a 3D replay alongside the measured timings. Each view should let the reviewer trace what they see back to the recorded numbers.
Once engineers have checked a feature, it enters the PO's inventory of candidates to combine and merge. The PO uses a view tuned to product decisions and can launch the same workflow for different feature combinations. For example, combine the localisation change with a new planner recovery strategy: does the robot resume its task smoothly, wait indefinitely or repeatedly stop and restart? A module can meet its specification while the combined behaviour reveals a side effect or an unforeseen error mode.
The PO decides which combinations deliver the intended behaviour and which discoveries become new backlog items. Engineers trace problems through the architecture and improve the specification, harness or implementation. This closes the wider human loop in the diagram.
Start small, then use the evidence to scale
Start with one behaviour and one meaningful scenario: sensor input is interrupted, the estimate becomes unavailable and the system recovers as specified. Let the PO create the work item, let engineers make the architectural specification and harness, and let the tooling return plots and results through the engineering gate before the PO reviews the behaviour.
Treat the first component as setup followed by several real change-and-review cycles. Record the human effort, test runtime, plots that made a decision easy and unresolved questions. That gives a better basis for the next component than assuming every team needs the same preparation.
So the answer to the tooling question at the start is a graphical interface for verification at every level. Engineers use plots and robot visualisations to check implementations and the tools that evaluate them. The PO uses views of the same evidence to judge combined product behaviour. Every accepted change should answer what was requested, what went in, what came out, how it was checked and who owns the decision.