Founder

Escribano

2025
Berlin, Germany

Local-first work memory for AI agents — on-device inference, retrieval, and a native MCP interface. Built end-to-end, from macOS app to protocol.

AI Infrastructure MCP On-Device Inference MLX Local-First Swift Open Source

Escribano

A private work memory for AI agents — local-first, MCP-native.

The problem

Coding agents are stateless. Every session starts from zero. Between sessions, your work context evaporates: the agent can see the repo, but it can’t see what you saw — what you searched, what you debugged, which tabs you had open, which decisions you made and why. The context that makes a senior engineer effective lives in the gap between the code and the human, and no agent has access to it. Escribano closes that gap by recording your work as it happens and making it queryable.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                  ANY MCP-COMPATIBLE AGENT                       │
│          (Claude Code, OpenCode, Cursor, custom agents)         │
├─────────────────────────────────────────────────────────────────┤
│              MCP SERVER (native Swift, stdio)                   │
│   escribano_search │ escribano_recent │ escribano_entities      │
│   escribano_status                                              │
├─────────────────────────────────────────────────────────────────┤
│                    RETRIEVAL (SQLite)                           │
│        FTS5 full-text index │ BM25 ranking │ filters            │
├─────────────────────────────────────────────────────────────────┤
│              ON-DEVICE UNDERSTANDING (MLX)                      │
│   Qwen3-VL (VLM inference) │ LLM summarization │ pHash dedup    │
├─────────────────────────────────────────────────────────────────┤
│                CAPTURE (ScreenCaptureKit)                       │
│         Privacy-filtered, opt-in screen recording               │
└─────────────────────────────────────────────────────────────────┘

Every layer runs on your machine. Nothing is sent anywhere.

Key decisions

On-device inference as the privacy wedge. Screen content is the most sensitive data a developer produces — credentials, customer data, private code. I decided nothing would ever leave the machine: VLM and LLM inference run on-device via MLX, PII redaction happens at the query layer before anything reaches an agent, and screenshots are strictly opt-in. This wasn’t just a feature choice; it was a deliberate market wedge. The people who most need work memory are the ones who can’t ship their screen to a third-party API.

MCP as THE agent interface. Rather than building integrations per-agent, the product exposes a single MCP server over stdio with four tools: escribano_search, escribano_recent, escribano_entities, and escribano_status. The MCP server is a thin transport adapter — all query logic lives in a shared core consumed by the CLI and the MCP server alike. Any MCP-compatible agent gets full access to your work memory with zero custom integration.

Port/adapter architecture. I benchmarked six different VLM approaches over the life of the project. Each swap — OCR to VLM, Ollama to MLX-VLM, capture backend changes — cost exactly one new adapter file and zero changes to business logic. When your substrate is moving as fast as local model inference, the architecture has to make swapping cheap.

The evolution

I first validated the pipeline as a TypeScript clean-architecture prototype — built with AI agents in about three weeks, zero architectural rewrites. Once the pipeline proved itself, I went native: a Swift daemon and macOS app, for direct ScreenCaptureKit access and MLX inference performance. This was deliberate product sequencing — validate fast in a flexible stack, then invest in native where it matters — not a rewrite born of failure.

The meta-story

Escribano was built 100% with AI agents. The full story — the documentation practices, multi-model strategy, and discipline that made it work — is in the companion post: The AI Doesn’t Replace Engineering Discipline. It Demands It.

Honest limits

  • Accuracy trade-off: on-device models are smaller than frontier cloud models. You trade some understanding accuracy for complete privacy. For most work-context retrieval that trade wins, but it’s real.
  • Apple Silicon only: MLX inference and ScreenCaptureKit tie the product to macOS on Apple Silicon.
  • Beta-stage: this is an early product. The pipeline works, the interface is stable, but expect rough edges and active iteration.
Visit Project