Inspect Repair Evidence
One Playwright test, written once against the original markup, run unchanged against four states of the same checkout
Refactored UI: PASS (valid heal)
4 selector(s) healed, all assertions passed
Plain Playwright, same test: FAIL · TimeoutError: locator.waitFor: Timeout 2000ms exceeded.
What the refactor changed in the compressed DOM
54 changed line(s) vs Original · 44.4KB → 1.7KB- <header class="top">- <span class="brand">Trailhead Outfitters</span>+ <header>+ <span class="cfac1ee">Trailhead Outfitters</span>- <button id="checkout-btn" data-testid="checkout-btn" class="btn primary" type="button">Checkout <span class="badge" id="cart-count">1</span>+ <button id="e156e1f" data-testid="t56f440" class="c559eb1 c701b44" type="button">Go to checkout <span class="c7df557" id="ec20cbd">1</span>- <section class="view on">- <div class="grid">- <article class="card">+ <section>+ <div class="c6f03d2">+ <article class="cb5c1cc">- <div class="price">$89</div>- <button id="add-to-cart-1" data-testid="add-1" class="btn primary" type="button">Add to cart</button>+ <div class="c395d1c">$89</div>+ <button id="e607ab3" data-testid="t50a42e" class="c559eb1 c701b44" type="button">Add to bag</button>- <article class="card">+ <article class="cb5c1cc">- <div class="price">$64</div>- <button id="add-to-cart-2" data-testid="add-2" class="btn primary" type="button">Add to cart</button>+ <div class="c395d1c">$64</div>+ <button id="eafd9ef" data-testid="tc5c822" class="c559eb1 c701b44" type="button">Add to bag</button>- <article class="card">+ <article class="cb5c1cc">- <div class="price">$29</div>- <button id="add-to-cart-3" data-testid="add-3" class="btn primary" type="button">Add to cart</button>+ <div class="c395d1c">$29</div>+ <button id="ec1bc1f" data-testid="t896bee" class="c559eb1 c701b44" type="button">Add to bag</button>- <section class="view">- <div class="panel">+ <section class="cb1f6e5">+ <div class="cfef97b">- <label class="field">Email <input id="email" name="email" type="email" placeholder="you@example.com" data-testid="email">+ <label>Email <input id="e7fa906" name="email" type="email" placeholder="you@example.com" data-testid="t806592">- <div class="row">- <button data-testid="back" class="btn" type="button">Back to cart</button>- <button data-testid="cancel" class="btn danger" type="button">Cancel</button>- <button id="place-order" data-testid="place-order" class="btn primary" type="button">Place order</button>+ <div class="ce857ca">+ <button data-testid="t0ac76b" class="c559eb1" type="button">Back to cart</button>+ <button data-testid="t7203bb" class="c559eb1 c60e758" type="button">Cancel</button>+ <div class="c592c61">+ <span>+ <button id="ee656ab" data-testid="t3e4210" class="c559eb1 c701b44" type="button">Complete purchase</button>+ </span>+ </div>- <section class="view">- <div class="panel">- <h2>Order confirmed <span id="order-no">+ <section>+ <div class="cfef97b">+ <h2>Order confirmed <span id="ecf8ff2">- <footer class="foot">© Trailhead Outfitters. Prices in USD. Free returns within 3…</footer>+ <footer class="cbd715d">© Trailhead Outfitters. Prices in USD. Free returns within 3…</footer>
Repairs applied during the run
#add-to-cart-3[data-testid="t896bee"]validated
Escalation ladder (cheapest first)
- L1 Smart Retry · miss · 545ms→
- L3 Multi-Selector Fallback · miss · 2ms→
- L4 DOM Geometry & Layout Matcher · selected · 50ms
- Confidence
- 79%
- DOM sent to repair
- 44.4KB → 1.7KB (26.8x)
- Ladder time
- 607 ms
- LLM cost
- $0.0000 (offline)
Why: best layout match: text, role, position and structure agree (text 0.5, role 1, attrs 1, position 0.999, size 0.993, context 1, intent 0.4)
5 assertion(s) passed after the repair
#checkout-btn[data-testid="t56f440"]validated
Escalation ladder (cheapest first)
- L1 Smart Retry · miss · 551ms→
- L3 Multi-Selector Fallback · miss · 6ms→
- L4 DOM Geometry & Layout Matcher · selected · 27ms
- Confidence
- 91%
- DOM sent to repair
- 44.4KB → 1.7KB (26.8x)
- Ladder time
- 590 ms
- LLM cost
- $0.0000 (offline)
Why: best layout match: text, role, position and structure agree (text 0.72, role 1, attrs 1, position 0.963, size 0.877, context 1, intent 1)
5 assertion(s) passed after the repair
#emailinput[name="email"]validated
Escalation ladder (cheapest first)
- L1 Smart Retry · miss · 539ms→
- L3 Multi-Selector Fallback · selected · 54ms
- Confidence
- 86%
- DOM sent to repair
- 44.4KB → 1.7KB (26.8x)
- Ladder time
- 597 ms
- LLM cost
- $0.0000 (offline)
Why: recorded alternate selector still identifies one element (2/3 alternates agree)
4 assertion(s) passed after the repair
#place-order[data-testid="t3e4210"]validated
Escalation ladder (cheapest first)
- L1 Smart Retry · miss · 539ms→
- L3 Multi-Selector Fallback · miss · 6ms→
- L4 DOM Geometry & Layout Matcher · selected · 21ms
- Confidence
- 71%
- DOM sent to repair
- 44.4KB → 1.7KB (26.8x)
- Ladder time
- 570 ms
- LLM cost
- $0.0000 (offline)
Why: best layout match: text, role, position and structure agree (text 0.075, role 1, attrs 1, position 0.948, size 0.824, context 0.833, intent 1)
4 assertion(s) passed after the repair
The test's own assertions (these decide whether a heal counts)
- URL navigated to the confirmation view
- order confirmation text is shown
- API state changed: exactly one order exists for jane@example.com
Raw evidence (JSON from the Playwright reporter)
{
"heals": [
{
"testTitle": "customer buys an insulated bottle",
"file": "demo/checkout.spec.ts",
"line": 27,
"url": "http://127.0.0.1:61255/?state=refactored",
"original": "#add-to-cart-3",
"healed": "[data-testid=\"t896bee\"]",
"action": "click",
"level": 4,
"levelName": "DOM Geometry & Layout Matcher",
"confidence": 0.7904999080680752,
"reasoning": "best layout match: text, role, position and structure agree (text 0.5, role 1, attrs 1, position 0.999, size 0.993, context 1, intent 0.4)",
"steps": [
{
"level": 1,
"name": "Smart Retry",
"status": "miss",
"reason": "still failing after 500ms: matches no element",
"ms": 545
},
{
"level": 3,
"name": "Multi-Selector Fallback",
"status": "miss",
"reason": "1 alternates tried, none resolves to a single visible element",
"ms": 2
},
{
"level": 4,
"name": "DOM Geometry & Layout Matcher",
"status": "selected",
"reason": "best layout match: text, role, position and structure agree (text 0.5, role 1, attrs 1, position 0.999, size 0.993, context 1, intent 0.4)",
"ms": 50
}
],
"domBytesRaw": 45439,
"domBytesCompressed": 1696,
"elapsedMs": 607,
"llmCostUsd": 0,
"verdict": "validated",
"verdictReason": "5 assertion(s) passed after the repair",
"assertionsAfter": 5,
"evidence": {
"text": 0.5,
"role": 1,
"attrs": 1,
"position": 0.999,
"size": 0.993,
"context": 1,
"intent": 0.4,
"score": 0.79,
"coverage": 1,
"selectorKind": "testid"
},
"timestamp": 1792168237134
},
{
"testTitle": "customer buys an insulated bottle",
"file": "demo/checkout.spec.ts",
"line": 28,
"url": "http://127.0.0.1:61255/?state=refactored",
"original": "#checkout-btn",
"healed": "[data-testid=\"t56f440\"]",
"action": "click",
"level": 4,
"levelName": "DOM Geometry & Layout Matcher",
"confidence": 0.9098848251115816,
"reasoning": "best layout match: text, role, position and structure agree (text 0.72, role 1, attrs 1, position 0.963, size 0.877, context 1, intent 1)",
"steps": [
{
"level": 1,
"name": "Smart Retry",
"status": "miss",
"reason": "still failing after 500ms: matches no element",
"ms": 551
},
{
"level": 3,
"name": "Multi-Selector Fallback",
"status": "miss",
"reason": "2 alternates tried, none resolves to a single visible element",
"ms": 6
},
{
"level": 4,
"name": "DOM Geometry & Layout Matcher",
"status": "selected",
"reason": "best layout match: text, role, position and structure agree (text 0.72, role 1, attrs 1, position 0.963, size 0.877, context 1, intent 1)",
"ms": 27
}
],
"domBytesRaw": 45439,
"domBytesCompressed": 1696,
"elapsedMs": 590,
"llmCostUsd": 0,
"verdict": "validated",
"verdictReason": "5 assertion(s) passed after the repair",
"assertionsAfter": 5,
"evidence": {
"text": 0.72,
"role": 1,
"attrs": 1,
"position": 0.963,
"size": 0.877,
"context": 1,
"intent": 1,
"score": 0.91,
"coverage": 1,
"selectorKind": "testid"
},
"timestamp": 1792168239762
},
{
"testTitle": "customer buys an insulated bottle",
"file": "demo/checkout.spec.ts",
"line": 40,
"url": "http://127.0.0.1:61255/?state=refactored#/checkout",
"original": "#email",
"healed": "input[name=\"email\"]",
"action": "fill",
"level": 3,
"levelName": "Multi-Selector Fallback",
"confidence": 0.862,
"reasoning": "recorded alternate selector still identifies one element (2/3 alternates agree)",
"steps": [
{
"level": 1,
"name": "Smart Retry",
"status": "miss",
"reason": "still failing after 500ms: matches no element",
"ms": 539
},
{
"level": 3,
"name": "Multi-Selector Fallback",
"status": "selected",
"reason": "recorded alternate selector still identifies one element (2/3 alternates agree)",
"ms": 54
}
],
"domBytesRaw": 45448,
"domBytesCompressed": 1696,
"elapsedMs": 597,
"llmCostUsd": 0,
"verdict": "validated",
"verdictReason": "4 assertion(s) passed after the repair",
"assertionsAfter": 4,
"evidence": {
"alternatesTried": 3,
"alternatesMatching": 2,
"similarity": 0.6
},
"timestamp": 1792168242444
},
{
"testTitle": "customer buys an insulated bottle",
"file": "demo/checkout.spec.ts",
"line": 41,
"url": "http://127.0.0.1:61255/?state=refactored#/checkout",
"original": "#place-order",
"healed": "[data-testid=\"t3e4210\"]",
"action": "click",
"level": 4,
"levelName": "DOM Geometry & Layout Matcher",
"confidence": 0.7125747272459301,
"reasoning": "best layout match: text, role, position and structure agree (text 0.075, role 1, attrs 1, position 0.948, size 0.824, context 0.833, intent 1)",
"steps": [
{
"level": 1,
"name": "Smart Retry",
"status": "miss",
"reason": "still failing after 500ms: matches no element",
"ms": 539
},
{
"level": 3,
"name": "Multi-Selector Fallback",
"status": "miss",
"reason": "3 alternates tried, none resolves to a single visible element",
"ms": 6
},
{
"level": 4,
"name": "DOM Geometry & Layout Matcher",
"status": "selected",
"reason": "best layout match: text, role, position and structure agree (text 0.075, role 1, attrs 1, position 0.948, size 0.824, context 0.833, intent 1)",
"ms": 21
}
],
"domBytesRaw": 45448,
"domBytesCompressed": 1696,
"elapsedMs": 570,
"llmCostUsd": 0,
"verdict": "validated",
"verdictReason": "4 assertion(s) passed after the repair",
"assertionsAfter": 4,
"evidence": {
"text": 0.075,
"role": 1,
"attrs": 1,
"position": 0.948,
"size": 0.824,
"context": 0.833,
"intent": 1,
"score": 0.713,
"coverage": 1,
"selectorKind": "testid"
},
"timestamp": 1792168245047
}
],
"blocked": []
}Copied from `python demo.py` (artifacts/demo) and `python scripts/run_benchmark.py` (BENCHMARK_RESULTS.json). Not hand-written. Generated 10/16/2026, 12:31:55 PM.