Concepts · new ↔ old

Everything TRIF changes, and why.

If you've written Selenium or Playwright, you already know most of this — under different names, with different trade-offs. Here's each TRIF concept, the traditional idea it maps to, and the benefit of the change.

At a glance

TRIF concept	Traditionally	Why it's better
Recipe	a test script / test case	human- and agent-readable, engine-portable, runs from ~400 tokens
Cookbook	a test project / suite	one versioned artifact; knowledge captured once, reused at O(1-test)
Anchor	a selector / locator	resilient semantic resolution; drift is named, not a mystery break
Oracle	assertions in the script	the success criterion is explicit and gradeable by anyone
Four verdicts	pass / fail	separates a real bug from drift from "couldn't run"
Binding & graduation	a compiled test script	pay to learn the app once, then rerun at $0 — script is disposable
Disclosure levels	load the whole framework	a test costs ~1,500 tokens no matter how big the app is
Freshness ledger	— (a wiki that rots)	the cookbook tells you what's stale before a run lies
Healing	manual maintenance / silent self-heal	repair is explicit and reviewable — never masks a regression
Four doors	framework / language lock-in	same tests from any agent, any CI, or by hand — no rewrite
Evidence & report	a report + screenshot-on-fail	native-grade, uniform evidence for every verdict and lane
IA map	— (tribal knowledge)	see exactly what's covered and how the app is modeled

In detail

Recipe

maps to → a test script / test case

A single test written as human-readable intent: a setup line, ordered steps in a small fixed verb grammar (goto · click · fill · read · wait_for …), and an explicit oracle. It refers to elements by anchor, never by raw selector, and lives in one small markdown file.

Was

A .spec.ts (or Java/Python) file: selectors, waits, and assertions interleaved in code, tied to one engine and one language, only a programmer can read.

Now

A recipes/TC-01.md: steps + oracle in plain language, engine-agnostic, ≤400 tokens — a person, an agent, or a compiled script can all execute it.

BenefitPortable and legible. The same test runs on Playwright, CDP, an agent, or by hand — and it's small enough to hand an LLM one at a time, which is what makes O(1-test) context possible.

Cookbook

maps to → a test project / suite

The versioned bundle of everything known about testing one app: its recipes, the app map + anchor registry, a healing playbook, and a freshness ledger — a directory <app>.trif/ with a semantic version.

Was

Specs + page objects + fixtures + config scattered across a code repo, with no signal about which parts still match the live app.

Now

One self-describing artifact. Explore the app once to write it; then any executor consumes it — no re-exploration, no framework setup.

BenefitKnowledge captured once, reused forever. The expensive part (learning the app) is paid a single time and versioned; every later run is O(1-test).

Anchor

maps to → a selector / locator

A semantic reference to a UI element — role + accessible name + a stable hint + a note on the non-obvious bits — resolved at run time by a ladder: engine hint first, then role/name, then an agentic fallback. Stored in map/anchors.json.

Was

A brittle string literal — #firstHeading, an XPath, a CSS chain — hardcoded in the test. When markup changes it breaks with a bare "element not found".

Now

An anchor that degrades gracefully: if the hint misses, it resolves by role+name; if that misses too, it's a named FAIL-STALE with a patch proposal — not a silent break.

BenefitResilience with a paper trail. The same anchor also compiles to a concrete locator(...) when a recipe graduates — so you get flexible resolution at author time and a fast selector at run time.

Oracle

maps to → assertions inside the script

The test's success condition, made first-class and explicit: deterministic checks (equals · contains · count · matches) are preferred; "judged" (model-graded) oracles are second-class and can never auto-file a bug.

Was

expect(...) calls buried among the actions, implicit and engine-specific — you have to read the code to know what "pass" even means.

Now

A dedicated oracle: block anyone (human, agent, compiled script) grades the same way — and deterministic is the path of least resistance.

BenefitYou can trust a pass. The criterion is legible and uniform across every lane, and fuzzy judgments are quarantined so they never masquerade as hard bugs.

Four verdicts

maps to → pass / fail

PASS · FAIL-BUG (real defect, evidence-quoted) · FAIL-STALE (the app drifted — an anchor miss) · BLOCKED (couldn't run: missing capability, unreachable app, auth).

Was

A red X. "App has a bug", "app changed", and "the runner fell over" all look identical — so reds get ignored and flakes get chased.

Now

Three failure modes named apart, under strict honesty rules (retry-once-before-fail; a bug needs quoted evidence; a miss is STALE, never a fake path).

BenefitReds you can act on. Bugs go to engineers, drift goes to healing, blocks go to infra — and "flaky test" stops being a catch-all excuse.

Binding & graduation

maps to → a compiled test script

When a recipe passes on the intent lane and is deterministic-ready, its steps + anchors + oracle compile into a real script (Playwright .spec.ts, Katalon .groovy) that reruns with no model. See exactly how in the infographic.

Was

You hand-write the script and own it forever — every UI change is your maintenance bill, and a stale script fails loudly or (worse) passes wrongly.

Now

The recipe is the source of truth; the script is a generated, disposable fast-path. A binding miss/stale/failure is only a signal — the run falls back to the recipe and heals.

BenefitPay to learn, not to re-run. The model cost is spent once; reruns are $0 and deterministic — and you never trust a stale script, because it can't return a verdict on its own.

Disclosure levels

maps to → loading the whole framework

The cookbook is layered (router → recipe → healing → map → ledger) so an executor loads only what one test needs: the router (≤1000 tokens) + one recipe (≤400). The repair layers load only on failure or authoring.

Was

Boot the whole test framework and page-object graph — or, with an AI agent, dump the entire app into context and pay for it every run.

Now

Progressive disclosure: zero-to-one-executed-test stays under ~1,500 tokens regardless of how large the app or cookbook is.

BenefitConstant cost, not linear. A 500-recipe cookbook runs one test as cheaply as a 5-recipe one — the thing that makes agent-run testing economical.

Freshness ledger

maps to → (nothing — a wiki that rots)

A dated coverage record (coverage.json) that tracks what's exercised and fingerprints each recipe + its resolved anchors, so staleness is detectable before a run.

Was

No first-class notion of "is this test still valid?" — you find out when it breaks in CI, or a doc drifts silently out of date.

Now

stale reports per-surface freshness; a binding retires automatically the moment its recipe or anchors change.

BenefitThe artifact stays honest. Drift is surfaced as data, not discovered as a surprise — and nothing runs on a fast-path it no longer earns. Deep dive →

Healing

maps to → manual maintenance / silent self-heal

When a run drifts, it emits a machine-readable patch proposal (anchor-fix, new-quirk, …). A maintainer — the only writer — applies it, bumps the version, and the recipe re-graduates. Executors stay read-only.

Was

Either a human hand-fixes selectors after every UI change, or "self-healing locators" silently adapt — which can quietly absorb a real regression.

Now

Repair is an explicit, reviewable diff; oracle-drift fixes even require a human signature. Drift is never silently absorbed.

BenefitMaintenance you can audit. The system proposes; a human decides — so healing speeds you up without hiding what changed. Deep dive: stale detection & healing →

Four doors

maps to → framework / language lock-in

One cookbook, four ways in: an Agent Skill (SKILL.md), an MCP server, the CLI, or the raw files. Each is a thin wrapper over the same files; results flow back into the same schema.

Was

A Java test runs in the Java framework; a Cypress test needs Cypress. Switching runners or driving from an AI agent means a rewrite.

Now

The same tests run from Claude Code / Cursor, from CI via the CLI, from a hosted assistant over MCP, or by a human reading the markdown.

BenefitNo lock-in. The format is the standard; the runners are interchangeable — the moment a door enriches its output beyond the files, it stops being the standard, so they can't drift.

Evidence & report

maps to → a report + screenshot-on-fail

Every run leaves a native bundle — Playwright trace, video, console + network logs, screenshots — and a self-contained HTML report that shows the verdict, the resolved lane, the step trace, the oracle checks, and the evidence.

Was

An engine-specific report, often just a stack trace and a screenshot captured only when something failed.

Now

Uniform, native-grade evidence for every verdict, whichever lane ran — and a report that's a pure function of the result records.

BenefitBelieve the verdict. "Confident-but-unchecked" is designed out: the report observes the artifacts, so a PASS is backed by a trace you can open, not a tool's say-so.

IA map

maps to → (tribal knowledge)

A generated, browsable map of the app's information architecture — surfaces × anchors × recipes, color-coded by freshness, with a detail drawer.

Was

What's covered, and how the app is modeled, lives in someone's head or a stale Figma/wiki diagram — invisible to a reviewer.

Now

One command renders the whole cookbook as a review artifact; a reviewer sees coverage and structure without reading a line of code.

BenefitLegibility. A cookbook stops being a black box — coverage gaps and the app model are visible at a glance.