Your tools will think it is the cloud.
Tesseract runs an OpenAI-compatible inference server on your Mac. Point any client with a base-URL field at localhost, and the same local models that power the app answer. This is the one paper on this site allowed to speak jargon.
abstract
The server exists because a good agent needs a good engine, and the engine turned out to be a product of its own. It speaks the chat-completions API, so existing SDKs, editors, and agent tools work by changing one line. The interesting part is what happens to context: a tiered RAM and SSD radix prefix cache means the tokens you already paid to prefill stay paid for, across requests, across conversations, and across app restarts of the tools that call it.
keywords: openai-compatible · tiered prefix cache · localhost only
theorem 6.1 (compatibility)
Change one line, keep your stack.
Proof. The server answers POST /v1/chat/completions on localhost, port 8321 by default, with streaming. Anything built against the OpenAI API, an SDK, an editor plugin, an agent framework, only needs its base URL changed. The models behind it are the same catalog the app runs, so your tools and your assistant share one brain.
in plain terms
- ·/v1/chat/completions
- ·streaming responses
- ·any OpenAI API client
theorem 6.2 (the cache)
Context you already paid for stays paid for.
Proof. Agents resend the same growing prefix on every turn, and on-device that re-prefill is the whole latency story. The radix cache keeps hot prefixes in RAM, spills older ones to SSD, and rehydrates from disk at up to 0.87 GB/s, so reusing a context costs roughly 50 times less than recomputing it.
in plain terms
- ·~50× cheaper context reuse
- ·up to 0.87 GB/s from SSD
- ·hit rates other stacks miss
theorem 6.3 (a good guest)
It shares the machine.
Proof. When your Mac needs room for other work, Tesseract steps aside: the oldest cache entries move quietly to disk and return the moment they are needed. And you can watch all of it happen. The app's Activity and Cache pages show every request as it runs and what the cache is holding, live.
in plain terms
- ·yields memory under pressure
- ·live activity view
- ·nothing leaves localhost
fig. 01: the tiered prefix cache. Only the new tokens are work; the rest is remembered, in RAM or on disk.
the endpoint
POST /v1/chat/completions · streaming
the address
localhost, port 8321 by default
the models
the same local catalog the app runs, ~3.5 GB to ~21 GB
the dashboards
Activity and Cache pages in the app show every request live
also on board
an /mcp endpoint lets local agents drive a real browser
§ endsee for yourself
The server ships inside the app and turns on with one switch. It binds to localhost, serves the models you already downloaded, and sends nothing anywhere.
next in the series: paper no. 01 · the companion →