Code search tool benchmark: rg, ast-grep, MCP, and what the data actually says

I started with a hypothesis. ast-grep, given pre-written rules and MCP error semantics, should beat ripgrep and the built-in Grep tool on both search accuracy and token cost. It is a reasonable hypothesis. It is also wrong on token cost, and wrong on accuracy for every query class but one.

I ran a controlled benchmark across 331 Claude Code sessions and four production codebases: two Java services on Spring Boot and Quarkus, a Rails service, and a React/TSX frontend. Of those, 97 sessions had active search activity, which is where the hit-rate and cost-per-found-block figures come from. Another 325 had usage data behind the token cost and model routing figures. Here is what I found.

The most actionable finding has nothing to do with rg or ast-grep. Opus costs 19× more per session than Haiku. Model routing is the dominant cost lever, and every search-tool optimization in this article sits inside a rounding error next to it. That number is in Table 12. Read it first if you only have time for one table.

Setup

Four repos, three query classes, two tools (rg and ast-grep with fixed rules), three rounds aggregated. The query classes are:

• Symbol definition: find where a class, method, or identifier is defined
• Exact string: find a literal string, log message, or config key
• Structural pattern: find all classes implementing an interface, all components with a given prop shape, all methods matching a call signature

Hit rate measures whether the tool returns any results at all on first call. Cost-per-found-block (Table 7) accounts for the load tax and reissue cost when zero results force a retry.

Table 1: Hit rate by repo × query class × tool

% hit on first call, 3 rounds aggregated. Higher = better recall.

rg wins 9/12, ties 2, loses 1. The single ast-grep win is React/TSX structural patterns, the one query class where rg returns 0% and ast-grep returns 66.7%. Across every other query class and every other repo, rg wins outright. That is a decisive result.

The one case where ast-grep is mandatory

The React/TSX structural pattern cell is not close. rg returns zero on every structural query against a JSX/TSX codebase. Not a low hit rate. Zero. JSX component structure, prop shapes, and hook usage patterns all need CST-aware matching. A regex over text cannot reliably find every component that uses a given hook when the hook call may be nested, renamed, or composed.

Two things make ast-grep work here and fail elsewhere:

1. Dump the CST before you write the rule. JSX component trees carry intermediate wrapper nodes you cannot see by reading the source. A has check then fails quietly whenever the target node sits deeper than expected. Running dump_syntax_tree(format: cst) on a minimal example first shows the real tree shape. Skip that step and your rules return zero, which looks exactly like "not found."

2. The language flag must match the file extension. Setting language: tsx on a codebase that is 98% .jsx returns zero across the board, with no parse errors and no warnings. Check the file extension distribution first. tsx and jsx are not interchangeable.

stopBy: end is also mandatory at every has level above the target. Without it, ast-grep checks only direct children and misses deeply nested nodes without saying so.

Table 2: Latency (CLI vs MCP, wall-clock median ms)

MCP wins on every repo, running 2-5× faster on wall-clock and staying under a second throughout. Latency is the real MCP advantage, ahead of both token cost and error semantics. The Rails service CLI call takes 2.5 seconds, which is slow enough to feel during a session, and MCP brings it under one second. Use MCP when wall-clock speed matters. Use Grep or Bash when token cost matters.

Token cost: MCP is not cheaper

The hypothesis that MCP + ast-grep would be cheaper on tokens is refuted by the data.

Table 3: Per-call invocation prompt size (tokens)

Table 4: Per-call response size (tokens)

Caveat: Bash+rg sample is Opus-only, mostly rg -l file-listing, not match-with-context. Honest comparison: rg and ast-grep are comparable when used the same way.

Table 5: Round-trip cost per call (invocation + response, tokens)

Table 6: Session-total cost (1,576 MCP load tax + N × round-trip)

MCP never crosses over. At every session size measured, MCP costs more tokens than Grep or Bash: median at 5 calls, p75 at 11, p90 at 27, and p99 at 79. The 1,576-token load tax at session start never amortizes. The MCP round-trip does beat Grep per call, 284 tokens against 324, but not by enough to cover that fixed cost. Optimize for token cost and Grep or Bash with rg is the right tool. Optimize for latency or structured error semantics and MCP is. They are different targets.

Table 7: Cost-per-found-block

iter = (100 / hit%) × (1 + zero% × reissue%). Accounts for the full cost of finding one result, including retries on zero. Lower = better.

rg wins 9 of 12 cells. An ∞ value means a 0% hit rate, so the tool cannot find the answer at any cost. ast-grep shows ∞ on two Java cells where it returned zero hits across every round. rg shows ∞ on JSX structural, the one confirmed ast-grep use case.

Table 8: Zero-result and rabbit-hole behaviour

MCP's structured error semantics do reduce the zero-result reissue rate. A zero result that arrives with a structured error lets the model recover more cleanly than an empty string does. The net effect is 4 to 6 percentage points: 20.5% effective reissue for MCP against 18.7% for Grep. That is small. The zero-result problem is mostly about routing by query class, and the tool interface barely moves it. Do not choose MCP for this reason.

Model routing: the number that dwarfs all of this

All of the above is search-layer optimization. Table 12 puts it in context.

Table 9: Search volume by model family

Opus searches roughly twice as often per session as Sonnet. Haiku searches not at all. It works only on paths the orchestrator supplies, through Read, Edit, Write, and known Bash commands. Haiku is a file-operation agent rather than a search agent, so routing file operations to it saves money without touching the search layer.

Table 10: Scope discipline by model

Scoping discipline is equal, with both models scoping about 90% of calls. The difference is result size. Opus pulls responses about 36% larger and reads about 50% more files per call. It is being more thorough on the same call, not less disciplined.

Table 12: Real per-session cost by model

From JSONL usage fields. Cache reads excluded (90% discount).

Opus costs 19× more per session than Haiku, and Sonnet costs 2.9×. The entire MCP load tax of 1,576 tokens is 0.27% of an Opus session's token budget. Tuning rg against ast-grep against MCP while defaulting to Opus for search-heavy tasks is rearranging deck chairs.

The practical implication is a routing rule. Search-heavy reasoning belongs on Sonnet. Haiku handles file operations on known paths at zero search overhead. Opus is reserved for architecture, security review, and high-stakes decisions where its reasoning quality earns the cost.

LSP: where it fits

Neither rg nor ast-grep handles semantic resolution: inheritance chains, type inference, or go-to-definition across file boundaries. That is LSP territory.

The routing rule is to use LSP for goToImplementation, hover, and type resolution. One caveat applies. LSP works in agent flow once its index is warm and persistent, as jdtls is for Java. Without a persistent cache, which is the common case for TypeScript, LSP starts cold every session and runs too slowly to use inline. For TypeScript semantic resolution, a structural ast-grep rule is the practical alternative.

Decision matrix

What the benchmark actually says

The original hypothesis held that ast-grep with MCP and rules would be the high-accuracy, low-token path. It is wrong on tokens and only partly right on accuracy. rg wins hit rate and cost-per-found-block in 10 or 11 of 12 cells across four codebases. ast-grep wins one cell decisively: JSX/TSX structural patterns, where rg returns zero.

MCP is worth using for latency, at 2-5× faster wall-clock, and for structured error semantics on zero results, worth a 4 to 6 point cut in the rabbit-hole rate. It is not worth using to save tokens, because it costs more at every session size measured.

The largest cost lever in this entire dataset is model selection. Opus costs 19× more per session than Haiku. Routing search-heavy work to Sonnet and file operations to Haiku has more token impact than any search tool choice.

There is no single right tool here. Route by query class. Verify language flags before writing ast-grep rules. Dump the CST before writing any structural rule. And read a zero result from either tool as a routing signal rather than a definitive answer.

The search routing logic, CST dump workflow, and model dispatch rules are part of the Agent Development Harness.