Contents
CodeNib: A Multi-View Data System for Serving Repository Context to Coding Agents
Why Coding Agents Struggle with Evolving Repositories
Modern coding agents — GitHub Copilot, Cursor, OpenHands, SWE-agent — are only as good as their understanding of the codebase they touch. But repositories do not sit still. A typical repo changes daily through commits, merges, and refactors. Every time an agent needs to answer a question about the code, it must re-run the same expensive ritual: grep for symbols, open files, walk call graphs, and re-derive structure it has already seen many times before. Per-task exploration is wasteful in both time and tokens.
Under the hood, this pain comes from a deeper architectural gap. Most tooling keeps its indexes fragmented: a text index here, a vector embedding store there, a language-server index somewhere else. None of these share state. When the repository moves forward, each index drifts in its own way, and the agent has no coherent picture of what changed. The result is duplicated work, stale answers, and a heavy token bill on every request.
CodeNib takes a fundamentally different angle: it treats the problem as a data-systems problem. In this framing, a commit is the base data, the derived views (lexical, dense, structural) are materialized query results, and each agent request is a query over those views. From this perspective, serving repository context stops being an ad-hoc guessing game and becomes a principled, incremental data pipeline.
What Is CodeNib?
CodeNib is an open-source, multi-view data system that builds three complementary views of a repository — lexical, dense, and structural — for each commit. Because the views are derived from a fork point and cached, agents can fetch rich context without re-analyzing the whole tree.
The system is made of three cooperating pieces: a repository view compiler that turns source into views, a view manifest that describes what each view covers and how fresh it is, and an agent runtime that serves these views to agents on request. Every output is mapped back to repository-relative source ranges, so the agent always knows exactly which file, scope, or callable a piece of context belongs to.
CodeNib is released under the Apache 2.0 license, requires Python 3.10 or newer, and ships as a v0.1.0 developer preview. Early recognition has been strong — it was ranked #2 Paper of the Day on HuggingFace.
The Multi-View Architecture
The design centers on three distinct views, each tuned for a different way an agent asks about code:
- Lexical view — a sparse retrieval index (BM25 / trigram) that matches keywords and identifiers quickly and cheaply.
- Dense view — a FAISS-backed embedding index that captures semantic similarity between natural-language queries and code.
- Structural view — a symbol graph built from SCIP / LSP that records callers, callees, and type relationships.
Each view is organized at three levels of granularity: L0 for whole files, L1 for scopes, and L2 for individual callables (functions and methods). Coarser levels are cheaper and faster; finer levels give the agent precise targets.
Coordinating all of this is the view manifest M_c, a per-commit record of which views exist, their status, and their capabilities. You can read the manifest as a lookup boundary: it tells the runtime, for a given commit, exactly what is available and how trustworthy it is. Crucially, the views are fail-isolated — if the structural graph fails to build, the lexical and dense views still serve correct results instead of taking the whole system down.
Incremental Maintenance: Update Instead of Rebuild
Rebuilding all views on every commit is prohibitively expensive as a repository grows. CodeNib instead updates views incrementally. Structural repair is assisted by Git history and the language server, so the symbol graph only patches the edges that actually changed. Vector embeddings are reused through content-addressed lookup — if a code unit did not change, its embedding is not recomputed.
This gives users a useful distinction: file-level updates handle coarse, low-frequency changes cheaply, while symbol-level updates fire when an individual callable changes and only that slice of the graph and index needs touching. The result is dramatically lower latency for real-world commits.
Getting started takes two commands. The first spins up the Wiki UI for browsing a repository in your browser:
pip install codenib
codenib wiki /path/to/repository
To serve the repository to agents over MCP, install the extra and run the MCP server:
pip install "codenib[mcp]"
codenib index /path/to/repository
codenib mcp /path/to/repository
Benchmark Results
CodeNib was evaluated across 100 snapshots from 25 repositories, drawing on SWE-Bench Verified and multilingual datasets, across five models: Claude Haiku 4.5, Qwen3.5-9B, Qwen3.5-27B, Gemma 4-12B, and Gemini 2.5 Flash. The headline numbers are compelling:
| Metric | Result |
|---|---|
| Graph update speedup (median) | 8.67x |
| Vector update speedup (median) | 25.44x |
| Graph transitions matching rebuild | 45.5% (15/33) |
| Vector transitions matching rebuild | 90.3% (28/31) |
| Static/live latency ratio (matched subset) | 4.72x |
| Static navigation match rate | 63.2% (632/1000) |
| Token savings (5 models) | 50-87% |
A word on how to read these numbers. The 8.67x graph speedup and 25.44x vector speedup are median speedups computed only on transitions whose output matched an independent rebuild — they are not overall throughput figures. Likewise, the 4.72x latency ratio applies only to the 632/1000 matched requests, not the entire navigation set. The systemic takeaway is clear: vector reuse (90.3% match) is far more reliable than graph repair (45.5% match), and the token savings of 50-87% are achieved while holding localization margin.
Try CodeNib Yourself
The fastest way to feel the difference is to point CodeNib at a repository and open the Wiki UI. Run pip install codenib, then codenib wiki /path/to/repository, and browse the source with linked context at localhost:3000. Every symbol you click resolves to a concrete source range, making exploration feel like reading documentation instead of guessing through grep.
For agents, install the MCP extra (pip install "codenib[mcp]"), index the repository with codenib index, then expose it with codenib mcp. Your agent framework can then consume the same views through standard MCP or LSP tooling, giving Copilot, Cursor, OpenHands, or SWE-agent a shared, reusable understanding of the codebase instead of a fresh re-discovery on every turn.
Limitations and Validity Boundaries
CodeNib is honest about where it stops. Static navigation does not fully replace a live language server. Reference resolution matched only 39% of cases, and definitions showed a 12.6% mismatch — so for tight, correctness-critical edits you should still confirm against a live LSP.
Second, graph expansion showed no clear statistical gain; the graph ablation remains unresolved, meaning the structural view is not yet proven to add measurable lift on its own. Third, patch generation is explicitly out of scope — CodeNib serves context, it does not synthesize fixes. These boundaries are stated plainly so you can rely on the system where it is strong and pair it with other tools where it is not.
Conclusion
CodeNib reframes repository context from a per-task exploration chore into reusable, incrementally maintained views. By treating commits as base data and requests as queries, it offers coders and coding agents a fast, token-efficient path to understanding even fast-moving codebases — while being transparent about its limits.
Future work points toward a concurrent heterogeneous repository database, post-training integration, and more resource-efficient serving. If you want to follow along or contribute, the project lives at github.com/sysevol-ai/CodeNib, and the full paper is at arxiv.org/abs/2607.25431.
