codeweb vs the way agents find code today
Every number on this page comes out of a file committed in this repository, produced by a harness you can re-run. Where a result did not replicate, this page says so and drops the claim.
The honest comparison is against what an agent does without a graph: it greps, reads what it found, and greps again. That baseline is measurable, so it is the one measured here.
0.44 → 0.74
of a function's real callers found by a frontier agent — grep baseline versus codeweb, same context budget.
bench/experiments/efficiency-pilot.reps5-v090.json
126×
smaller measured context size — one call versus a simulated graph-assisted grep loop. This context-size comparison does not measure total agent-session token savings.
bench/results/oracle-ab.json
0 disagreements
across 497,864 comparisons of codeweb's answers against independent implementations.
bench/results/correctness-query.json
More of the callers, at the same cost
A paired A/B on a caller-discovery benchmark, engine frozen, the treatment arm reading the same budgeted responses agents actually receive over MCP.
Recall improved by +0.310 ± 0.039, with all 5 repetitions positive — roughly eight times the run-to-run noise. In absolute terms the grep arm found 0.44 of the real callers and the codeweb arm found 0.74.
Precision moved with it, +0.234 ± 0.080: fewer wrong callers, not more. The mechanism is deterministic rather than persuasive — fixing import-alias and member-access resolution is what surfaced the callers, and no model was involved in finding them.
Labelled preliminary, because grading is strict at the symbol level and one target (a Python import-edge gap) was an honest per-task loss. Both facts are in the receipt.
Receipt: bench/experiments/efficiency-pilot.reps5-v090.json · engine v0.9.0
The null published beside it
Token spend and tool calls were a wash: total tokens −84k ± 381k and tool calls +0.95 ± 3.6, both swamped by their own variance.
So the claim is completeness, not savings: the same spend buys more of the callers. An earlier run on a different base model did show large token and step savings; on the current, more frugal agent those did not replicate, so they are not claimed here.
Receipt: bench/experiments/efficiency-pilot.usage-v090.json
One call, or a loop of reads
"What breaks if I change this?" is the question that decides whether an agent's edit is safe, and it is the most expensive one to answer without a graph.
| Answering "what breaks if I change this?" | Grep loop | codeweb | Receipt |
|---|---|---|---|
| Rounds of tool calls | 7.2 | 1 | oracle-ab.json |
| Bytes injected into the agent's context | 153,274 | 1,214 | oracle-ab.json |
| Mean blast radius on the sampled targets | 21 | 21 | oracle-ab.json |
The grep loop is simulated generously — it is handed the graph's own frontier to expand, which a real one would have to discover. Even so, the context cost differs by a factor of 126.
Tokens are approximated as bytes over four. The cost being compared is what each channel injects into the agent's context.
Where each approach stops
Three ways to give an agent structural context. The rows are capabilities, and the codeweb column is the only one this repository can produce receipts for.
| Question | Grep / text search | One-hop symbol lookup | codeweb |
|---|---|---|---|
| Direct callers of a symbol | Partial — name collisions and misses | Yes | Yes, graded against raw edge sets |
| Transitive impact of an edit | Only by looping | Only by looping | One call, 0 disagreements vs an independent walk |
| Duplicated logic across domains | No | No | Body-confirmed, plus renamed clones |
| Dead code, ranked by confidence | No | Per symbol, if you know to ask | Whole-graph, safe tier precision 1.0 |
| A verdict CI can fail a pull request on | No | No — nothing to diff | Yes, from two graph snapshots |
| Same input, same answer | Yes | Yes | Yes — 1 digest per repo over 20 runs |
The "one-hop symbol lookup" column describes the capability class, not any particular implementation. The LSP FAQ covers that comparison in full. Receipts for the codeweb column: bench/results/correctness-query.json, bench/results/detection-accuracy.json, bench/results/determinism.json, bench/results/auxiliary.json.
What a hosted gate costs
The verdict is a deterministic graph diff, so a busy week is not an expensive week — for us or for you.
Flat, per active author
codeweb Teams is planned at €10 per active author per month, flat: unlimited repositories, pull requests, and gate runs. It is an intent ratified in the charter, not a live offer.
Everything that runs on one laptop against one repo stays free forever — the boundary says exactly where the line sits.
The category anchor
The competitive scan committed in this repository recorded commercial code-analysis gates priced at €18–27 per active author per month. Pricing under the category we are entering is deliberate.
Receipt: reports/COMPETITIVE.md. No claim here describes any specific competitor.
What we deliberately don't claim
This list is the same one the research page publishes, rendered from the project's claim ledger — including the rule that governs this very page.
- "Finds all clones / 100% recall" → High-precision, body-confirmed overlap on the clones it surfaces (F1 1.0 synthetic, 0.98 on real axios) — it deliberately drops ambiguous edges because a false positive costs more than a miss.
- "Replaces human review" → It produces candidate lists and shows which merges are cycle-safe to apply — a structural pre-flight, not a semantic verdict.
- "Agents provably edit better" → It improves caller-discovery recall (+0.31, all reps positive) at equal token cost on the budgeted responses agents receive (v0.9.0). An earlier run on a different base model also showed ~34% fewer steps; that savings did not replicate, and we say so. The full edit-quality A/B (H18) was a null on clean tasks — a floor effect, reported honestly.
- "100% deterministic" → Byte-deterministic, set-valued output — one digest across 20 runs per repo for nodes, edges, domains, and overlaps.
- "Sub-second every time" → Typical query ~43–50 ms; worst-case p95 51.9 ms on 3,215 symbols, where Node startup (~29 ms) dominates.
- "Beats commercial tools" → We compare only to stated, reproducible baselines (name-matching, lexical, random, the legacy fallback) — no head-to-head against tools we can't reproduce.