What This Skill Does
When you benchmark an AI coding agent, you're measuring the model — not the harness it runs inside. This skill gives you a five-dimension evaluation framework to assess what the harness actually contributes to performance, so you can select platforms on structural fit rather than leaderboard scores.
Problems It Solves
Model-benchmark conflation — the same model can score nearly double on identical tasks depending on which harness it runs inside. Published benchmarks compare weights, not environments, so they cannot predict real-world performance for your team.
Harness invisibility — execution environment, memory architecture, context management, tool integration, and multi-agent coordination are almost never surfaced in comparisons, yet each is a performance multiplier independent of model quality.
One-size-fits-all selection — harnesses embody fundamentally different philosophies ("collaborator at the desk" vs. "contractor in a clean room"). Treating them as interchangeable wrappers leads to structural mismatches that no prompt engineering can fix.
No re-evaluation cadence — teams that evaluate once lock in on a harness whose capabilities have since been overtaken. This skill includes an explicit anti-pattern for static evaluations.
What You Get
A structured assessment across five architectural dimensions, each with a decision table and targeted assessment questions:
Execution Philosophy — local/composable vs. isolated/cloud, and what that means for tool access and trust boundaries.
State & Memory — artifact-based session memory vs. repo-as-memory, and the documentation investment each requires.
Context Management — compaction and sub-agent delegation vs. sandbox isolation, and which fits deeply interconnected vs. parallel-independent tasks.
Tool Integration — filesystem-based skills with MCP support vs. server-mediated RPC, and the token cost and composability trade-offs of each.
Multi-Agent Architecture — orchestrated collaboration with task dependency tracking vs. git-coordinated isolation, and the cascade risk vs. safety trade-off.
You also get a fill-in scoring template that produces a structured HARNESS DIMENSION ASSESSMENT with explicit mismatch flags and a use/avoid/conditional recommendation.
Who Should Use This
Engineering leads and platform architects evaluating whether to adopt or switch AI coding agent platforms.
Teams whose current agent is underperforming relative to benchmark expectations and need to diagnose whether the gap is model or harness.
Organizations making procurement decisions based on published model comparisons who need a framework that reflects real deployment conditions.