More code ships; verification still waits

Code generation got cheap. Code verification didn’t. A coding agent can emit a thousand-line diff in minutes; a human reviewer needs an hour to read it properly, and a test suite that predates the agent can only check what someone thought to specify. The result is an asymmetry: throughput of change is exploding while confidence per change is flat or falling.

Test authoring can become another bottleneck as code changes faster. Agents can help draft scenarios and investigate unfamiliar pages, but teams still need to review assertions and establish that each test runs reliably.

TASK

Plain-English browser task

Describe the behavior and the success condition.

intent first
PROPOSAL

Playwright spec draft

The export includes locators, assertions, and a clean starting state.

diff to inspect
REVIEW

Run the reviewed spec

The committed test decides pass or fail without an inference call.

human decision
Review the generated test before CILet an agent propose the task or repair, then keep the executed check repeatable and reviewable.

Review generated tests before running them in CI

The design choice that separates serious systems from demos is where the model sits. LLMs are excellent at the parts of testing that benefit from judgment: reading an unfamiliar app, mapping flows, proposing assertions, diagnosing why a selector broke. They are terrible at the parts that need repeatability: deciding pass/fail on Tuesday the same way they did on Monday.

So put the model at authoring and healing time, never in the runtime path:

  • Authoring: an exploration agent crawls your app and emits real Playwright specs from plain-English scenarios — the mechanics we detailed in AI Playwright test generation.
  • Healing: when a locator breaks, the model proposes a patch against the new DOM; a human or policy layer approves it — the discipline described in self-healing test automation.
  • Runtime: the emitted spec executes deterministically. Same input, same verdict. No inference call decides whether your checkout test passes.
Code example
# Runtime path contains zero LLM calls:
#   spec (committed) → browser → assertions → pass/fail
# Model touches only:
#   crawl → generate → propose heal → human review → commit

Trust rule: any system where a probabilistic component decides pass/fail at runtime cannot be a merge gate. Determinism isn’t a nicety here — it’s what makes a red check mean something.

Coding agents should verify their own work

The most underrated shift is protocol-shaped. Via MCP, a coding agent doesn’t just write the feature — it invokes the QA platform itself, gets structured results back, reads the failure evidence, fixes the code, and re-runs. No human relay in between:

Code example
molar verify "the login form is visible" --url http://localhost:3000 --json

Structured results let a coding agent inspect a failed check without a developer manually copying screenshots. Molar’s CLI and MCP interfaces make browser checks available from the development workflow. The app and required test access must be configured first.

Decide which behavior the test must protect

Agents are converging on execution. Three things remain yours:

  • Intent. What should be true? Which flows matter enough to gate a merge on? An agent can discover flows, but only you know which ones would end your business if they broke.
  • Judgment calls. Is this pixel drift acceptable? Is this edge case a bug or a feature? These are product decisions wearing engineering costumes.
  • Review of proposed changes. Including proposed test changes — a healing patch that quietly weakens an assertion is a vulnerability, not a convenience.

The role shift is real but familiar: less time authoring steps, more time specifying outcomes and auditing what the machines proposed. It mirrors what happened to compilers — nobody misses hand-writing assembly, and nobody reviews the machine code either.

The failure mode worth designing against

Every automation wave in testing has produced the same pathology: tests that pass while the product breaks, because the suite was optimized for green rather than truth. Agent-driven QA can amplify this dramatically — an agent that both writes and heals its own tests can, without any malice, converge on assertions that match whatever the app currently does.

Three structural defenses: keep runtime verdicts deterministic so healing requires a reviewable diff; version scenarios as source files so weakening an assertion shows up like any other change; and sample agent-modified suites against known-broken builds periodically to confirm they still catch what they claim to catch. Mutation-style spot checks are cheap insurance here — break the checkout total on purpose once a month and confirm something red appears within one run.

The verification model to build

A useful direction is to version plain-English scenarios alongside code, review generated changes, and rerun tests from clean state. Humans still decide what behavior matters and whether an unexpected change is acceptable.

To evaluate Molar, inspect a recorded browser task and then choose one check for your app. Gxported Playwright tests are in beta; review each one before it runs in CI.