XERJ

Which memory or search tool do I need?

A ladder. Start at the bottom. Step up when the rung below stops fitting. Document count grows from tens to millions as you climb.

1. 1

   A notes or memory file

   School: a sticky note on the monitor.

   YAML or Markdown the agent re-reads every turn. Tiny, zero setup. Best for preferences and a handful of facts.

   Step up when that file is too long to paste every time.

2. 2

   Grep or reading files

   School: flipping through every textbook.

   Fine for a small repo. Each question still pulls whole files into context, so cost explodes as the tree grows.

   Step up when grep hits become a token bill.

3. 3

   XERJ

   School: the index at the back of the book.

   One index over a big, mixed folder. The agent asks a question and gets the exact passage with file:line. BM25, vectors, and hybrid on the same binary. Elasticsearch clients still work.

   Step up when you need a multi-user production cluster with someone else operating it.

4. 4

   A hosted vector database or managed Elasticsearch

   School: the library building, with staff.

   For multi-user production apps, SLAs, and ops you do not want to run. Qdrant Cloud, Weaviate, Pinecone, and managed Elasticsearch live here — compare hosted vector pricing.

   Stay here when many users share one cluster and auth, backups, and a vendor page matter.

Cost per question (output tokens)

From their reference-coding study: 8 tasks, 16 runs per arm, real billed output tokens. Source: xerj.org/case-studies/reference-coding.html and the GitHub README, checked 19 Oct 2026. Their caveat: the win is on private, niche, or new code, and it is neutral or worse on popular libraries the model already knows.

• From memory only 260,916 tokens · $11.18 · 11/16 solved
• Grep-driven agent 26,477 tokens · $3.27 · 16/16 solved
• XERJ 9,982 tokens · $1.58 · 16/16 solved · 2.7x fewer than grep

Use cases

• Reference coding. Index the nearest open-source repos and retrieve how they solved it before writing code. That is the measured 2.7x token cut.
• Codebase Q&A / RAG. Index a repo, ask for the mechanism, get the passage with file:line instead of a directory listing.
• A large knowledge base. Docs, PDFs, DOCX, slides, spreadsheets, and SQLite in one autoindex — no pipeline to configure.
• Log analysis. Mixed log formats become typed indices with Elasticsearch-shaped aggregations.
• Agent memory. /_memory/{namespace} stores what an agent learns and recalls it by meaning next session, plus a second-brain link graph.
• Replacing Elasticsearch. Same wire protocol, one binary. Test the queries you actually run; the four published losses versus Elasticsearch 8.13.4 are read p99 under heavy concurrent writes.

When not to use it

• Small memory sets: a sticky-note YAML or Markdown file the agent reads every turn is enough for preferences and facts.
• Popular public libraries the model already knows. Their study calls that case neutral or worse than grep.
• Shared multi-user hosts without auth set up. --insecure disables TLS and API-key auth. That is one user on localhost, not a team box.

Quick start

--insecure turns off TLS and auth. Every caller that can reach the port is a superuser. Use it only for one user on localhost. Auth is on by default otherwise — pass --api-key or XERJ_API_KEY from <data_dir>/admin.key.

curl -fsSL https://xerj.org/get | sh xerj --insecure --data-dir ./data & xerj init xerj autoindex ~/my-project xerj search "how do we rotate the WAL segment"

Windows PowerShell: irm https://xerj.org/get.ps1 | iex. Hand-verified install notes are on the README.

FAQ

What is XERJ?

XERJ is local AI search: one Apache-2.0 Rust binary that autoindexes a mixed folder (code, docs, PDFs, logs, spreadsheets) and answers with the exact passage plus file:line. It speaks the Elasticsearch REST API (1,366 of 1,369 conformance cases), so existing ES clients and Kibana work. Agent memory lives at /_memory/{namespace}, and xerj mcp exposes 13 tools. Neural embeddings are opt-in (~90 MB download); the default embedder is lexical and offline.

When should I use a sticky-note memory file or grep instead?

A YAML or Markdown notes file is enough for preferences and a handful of facts — tiny, zero setup. Grep or reading files is fine on a small repo. Step up to XERJ when the corpus is a big mixed folder and grep's token cost explodes. Skip XERJ on popular public libraries the model already knows: their own study calls that case neutral or worse.

Does it replace Elasticsearch?

It speaks the Elasticsearch REST API, so many ES clients keep working. Head-to-head against Elasticsearch 8.13.4 they report 55 wins, 26 ties, and 4 losses (the losses are read p99 under heavy concurrent writes), with 1.72x ingest and a 1.61x smaller on-disk footprint. It is not a fork of Elasticsearch internals. A hosted multi-user cluster still belongs on managed Elasticsearch or OpenSearch — /compare/vector-databases for hosted vector pricing.

Is --insecure safe?

No, not on a shared host. --insecure turns off TLS and API-key auth, so every caller that can reach the port is a superuser. The README says that is for one user on localhost. Auth is on by default for every start without --insecure.

Do I need neural embeddings or Jev?

No. The default embedder is lexical and fully offline. Neural embeddings are opt-in and download about 90 MB. Optional rerank through Jev from TypeSafe AI sends result text to TypeSafe and stays off unless a key is set. A local /_decide endpoint does label lookups with nothing leaving the machine.