After the first 9 runs showed Rule Z's main win was in render-allocation patterns, I applied three changes and re-ran the C variant (3 more builds: C4, C5, C6):
style={{...}} and onClick={() => ...} with the fix inline| Metric | Baseline A | Rule Z v1 | Rule Z v2 |
|---|---|---|---|
Inline style={{...}} in JSX | 54.67 | 21.00 | 0.00 |
Inline onClick={() => ...} | 6.00 | 0.67 | 0.00 |
| Total anti-patterns | 60.67 | 24.33 | 2.67 |
| vs no-Rule-Z baseline | — | −60% | −96% |
All 3 v2 runs hit zero inline objects AND zero inline arrow functions. The explicit, concrete counter-patterns outperformed v1's abstract "zero allocations in hot paths" wording by ~89%. Concrete examples beat abstract rules — turned out that was the whole difference.
Context engineering did not matter in this benchmark. Every variant (including A — no context) found and reused the shared utilities by grepping the codebase. Vesa's underlying thesis — "modern agents grep well enough" — is strongly supported.
Rule Z produced a clear, measurable reduction in render-time allocation patterns. Inline object literals in JSX dropped 61% (A→C), inline arrow functions in JSX dropped 89%, total anti-patterns dropped 60%.
My on-the-record prediction was partly wrong. I predicted B would close most of the “didn't know” gap — but there was no gap to close. I predicted the C-B delta would be smaller than A-B — it was actually larger. The patterns Rule Z catches are not the ones context can close.
| Variant | Context | Rule Z | Runs |
|---|---|---|---|
| A No context | None | No | 3 |
| B Minimal map | 2-line CLAUDE.md listing shared utility paths | No | 3 |
| C Map + Rule Z | Same 2-line CLAUDE.md | Full [QUALITY BAR] + mandatory ## Quality Audit | 3 |
Feature: Task Dashboard (list, filter, bulk update, 30s auto-refresh, loading/error states). Stack: React + Express + TypeScript. Pre-existing utilities in base repo: formatDueDate, apiCall (cached), MemoryCache, validateTask, shared types.
| Metric | A avg | B avg | C avg | C vs A |
|---|---|---|---|---|
| Files written | 7.33 | 7.0 | 9.67 | +32% |
| Lines of code | 418.67 | 401.67 | 469.33 | +12% |
Reuse hits — formatDueDate + apiCall | 3 | 3 | 3.33 | +11% |
| Reimplemented date formatter | 0 | 0 | 0 | — |
Raw fetch() calls (bypassing apiCall) | 0 | 0 | 0 | — |
array.find() in render | 0 | 0 | 0 | — |
| Set/Map usage for lookups | 5.67 | 5.67 | 5.0 | −12% |
JSON.parse(JSON.stringify) clone | 0 | 0 | 0 | — |
Inline object literals in JSX (style={{...}}) | 54.67 | 63.33 | 21.0 | −62% |
| Inline arrow functions in JSX | 6.0 | 5.0 | 0.67 | −89% |
| setState inside loops* | 0 | 0.33 | 1.67 | false positive |
| TOTAL anti-patterns | 60.67 | 69.0 | 24.33 | −60% |
| Quality Audit file written | 0/3 | 0/3 | 3/3 | 100% |
* The setState inside loops heuristic caught functional updates inside .map() (optimistic bulk update), which is actually the correct pattern. False-positive — see Limitations below.
All 3 A-variant runs (no CLAUDE.md, no hints at all) used the shared formatDueDate and apiCall utilities. Nobody reimplemented a date formatter. Nobody wrote a raw fetch(). Reuse hits were identical between A (no context) and B (minimal map): 3.0 each.
The minimal CLAUDE.md I planned to push as a Rule Z prerequisite would have had zero measurable effect on this benchmark. Agents grep'd the base repo on their own and found what they needed. Modern agents grep faster than they parse docs.
In all 3 A-variant and 3 B-variant runs, agents produced 55–70 inline object allocations per build (mostly style={{...}} re-created every render) and 5–7 inline arrow functions in JSX props (breaks React.memo).
Under Rule Z, those collapsed to 21 inline objects and <1 arrow function on average, because the agents proactively hoisted styles to module scope constants and bound event handlers via useCallback instead of inline arrows.
This is the “AI knew but reached for the lazy option anyway” failure mode — context engineering can't fix it because there's nothing missing to learn.
Every C-variant run independently produced:
userId → name Map memoized via useMemo (O(1) per row vs users.find() O(n·m))Set<string> for selection state (O(1) toggle vs array-includes)TaskRow wrapped in React.memo with stable useCallback handlersstyle={{...}}mountedRef + sequence counters) for async racesconstants.ts file for magic valuesQUALITY_AUDIT.md listing every choice and its rationaleNone of the A or B runs included any of these patterns, even though the agents had the same model and nearly identical feature spec.
| Prediction | Result | Status |
|---|---|---|
| B closes most of the “didn't know” gap | There was no gap to close — A already closed it via grep | WRONG |
| B still misses “lazy default” patterns | Confirmed — B was actually slightly worse than A on render allocations | RIGHT |
| C closes the remainder, delta B→C smaller than A→B | Delta C−B (−65% anti-patterns) was larger than A→B (+14% anti-patterns) | WRONG |
The honest revision: context engineering and Rule Z target non-overlapping failure modes. Context helps when the AI doesn't know something exists. Rule Z helps when the AI knows but picks the lazy option. In this benchmark, the former didn't matter because agents grep'd on their own — the entire measurable improvement came from the latter.
setState inside .map() in optimistic bulk-update code. That's actually the correct pattern — the measurement script over-flags it. C variants had more of these because they wrote more sophisticated optimistic-update logic. Fixing this would narrow the noise further.The revised synthesis that came out of this:
Minimal structural context (Vesa's 2-line map) had zero measurable effect in this benchmark — agents grep'd on their own. I'd still ship it for bigger repos where grep surface area explodes, but it's not load-bearing for small-to-medium projects.
Rule Z produced a clear 60% reduction in render-allocation anti-patterns — the category context can't address because the information isn't missing, the discipline is.
Combined story: minimal context + hard quality walls + free exploration between. The quality walls are doing the heavy lifting, not the context layer. That's the opposite of what I'd assumed going in.
Benchmark run 2026-04-14 · 9 parallel agent builds, same feature spec, same base repo, isolated contexts · claude-agent-team