# Table of Contents - [Introduction | Headroom](#introduction-headroom) - [Agent Orchestration | Headroom](#agent-orchestration-headroom) - [Agno | Headroom](#agno-headroom) - [Anthropic SDK | Headroom](#anthropic-sdk-headroom) - [API Reference | Headroom](#api-reference-headroom) - [AutoGen | Headroom](#autogen-headroom) - [Architecture | Headroom](#architecture-headroom) - [Benchmarks | Headroom](#benchmarks-headroom) - [Reversible Compression (CCR) | Headroom](#reversible-compression-ccr-headroom) - [Cache Optimization | Headroom](#cache-optimization-headroom) - [CI/CD Flow Diagrams | Headroom](#ci-cd-flow-diagrams-headroom) - [Claude Code on Azure AI Foundry | Headroom](#claude-code-on-azure-ai-foundry-headroom) - [Claude Code on Vertex AI | Headroom](#claude-code-on-vertex-ai-headroom) - [Code Compression | Headroom](#code-compression-headroom) - [Recover Codex State | Headroom](#recover-codex-state-headroom) - [Configuration | Headroom](#configuration-headroom) - [Context Management | Headroom](#context-management-headroom) - [CrewAI | Headroom](#crewai-headroom) - [Docker-Native Install | Headroom](#docker-native-install-headroom) - [Failure Learning | Headroom](#failure-learning-headroom) - [Error Handling | Headroom](#error-handling-headroom) - [Image Compression | Headroom](#image-compression-headroom) - [How Compression Works | Headroom](#how-compression-works-headroom) - [Filesystem Contract | Headroom](#filesystem-contract-headroom) - [Grok Build Integration | Headroom](#grok-build-integration-headroom) --- # Introduction | Headroom Introduction ============ Headroom is the context optimization layer for LLM applications. Compress tool outputs, DB results, file reads, and RAG results before they reach the model. Same answers, fraction of the tokens. Copy MarkdownOpen Headroom compresses everything your AI agent reads -- tool outputs, database results, file reads, RAG retrievals, API responses -- before it reaches the LLM. The model sees less noise, responds faster, and costs less. [Quick preview](https://docs.headroomlabs.ai/docs#quick-preview) ----------------------------------------------------------------- TypeScriptPython import { compress } from 'headroom-ai'; const messages = [\ { role: 'user' as const, content: 'Analyze these results' },\ ]; const result = await compress(messages, { model: 'gpt-4o' }); // compressionRatio is tokensAfter / tokensBefore, so savings is 1 - ratio. console.log(`Saved ${result.tokensSaved} tokens (${((1 - result.compressionRatio) * 100).toFixed(0)}%)`); from headroom import compress from openai import OpenAI messages = [{"role": "user", "content": "Analyze these results"}] result = compress(messages, model="gpt-4o") client = OpenAI() response = client.chat.completions.create( model="gpt-4o", messages=result.messages, ) print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})") [What gets compressed](https://docs.headroomlabs.ai/docs#what-gets-compressed) ------------------------------------------------------------------------------- | Content type | What happens | Typical savings | | --- | --- | --- | | JSON arrays (tool outputs) | Statistical analysis keeps errors, anomalies, boundaries | varies with array size and redundancy | | Source code | AST-aware compression preserves signatures, collapses bodies | opt-in; disabled by default | | Build/test logs | Keeps failures and errors, drops passing noise | varies with log verbosity | | Search results | Ranks by relevance, keeps top matches | varies with result set size | | Plain text | ModernBERT token classification removes redundancy | varies with redundancy | | Git diffs | Preserves change hunks, drops unchanged context | varies with diff size | | Images | ML router selects optimal resize/quality tradeoff | varies with image content | Savings depend heavily on how repetitive the content is -- see [Benchmarks](https://docs.headroomlabs.ai/docs/benchmarks) and the reproducible scenario table below for measured numbers on real workloads. [Where Headroom fits](https://docs.headroomlabs.ai/docs#where-headroom-fits) ----------------------------------------------------------------------------- Your Agent / App | | tool outputs, logs, DB reads, RAG results, file reads, API responses v Headroom <-- proxy, Python library, TS SDK, or framework integration | v LLM Provider (OpenAI, Anthropic, Google, Bedrock, 100+ via LiteLLM) Headroom works as a **transparent proxy** (zero code changes), a **Python function** (`compress()`), a **TypeScript function** (`compress()`), or a **framework integration** (LangChain, Agno, Strands, LiteLLM, Vercel AI SDK, MCP). [Real-world results](https://docs.headroomlabs.ai/docs#real-world-results) --------------------------------------------------------------------------- Headroom is built for the scenario that matters most in agent workflows: a single critical error or fact buried inside a large, mostly-routine tool output. SmartCrusher preserves error items, anomalies (values outside normal statistical range), and first/last boundaries by construction -- not by keyword matching, but by statistical analysis of field variance -- so the signal survives even when the surrounding noise is compressed away aggressively. Measured on local, seeded test data built from real MCP server output formats. These are not production telemetry, and they are reproducible: | Scenario | Before | After | Savings | | --- | --- | --- | --- | | Code search (100 results) | 17,199 | 13,597 | **21%** | | SRE incident debugging | 55,957 | 24,340 | **57%** | | Codebase exploration | 58,801 | 33,895 | **42%** | | GitHub issue triage | 46,067 | 32,429 | **30%** | uv run python benchmarks/index_proof_table.py --seed 20260902 Token counts are from the `gpt-5.6` tokenizer. The corpus is generated, so the exact before/after counts move slightly with the seed; the savings hold across seeds (Code search 21%, SRE 56--57%, triage 29--30%, exploration 41--50% over five seeds). Savings depend on how repetitive your tool output is, so treat these as a shape, not a promise. See [Benchmarks](https://docs.headroomlabs.ai/docs/benchmarks) for the wider suite. [Key Features](https://docs.headroomlabs.ai/docs#key-features) --------------------------------------------------------------- ### Lossless Compression (CCR) Compresses aggressively, stores originals, gives the LLM a tool to retrieve full details. Nothing is thrown away. [Learn more →](https://docs.headroomlabs.ai/docs/ccr) ### Smart Content Detection Auto-detects JSON, code, logs, text, diffs, HTML. Routes each to the best compressor. Zero configuration needed. [Learn more →](https://docs.headroomlabs.ai/docs/how-compression-works) ### Cache Optimization Stabilizes prefixes so provider KV caches hit. Tracks frozen messages to preserve the 90% read discount. [Learn more →](https://docs.headroomlabs.ai/docs/cache-optimization) ### Image Compression 40-90% token reduction via trained ML router. Automatically selects resize/quality tradeoff per image. [Learn more →](https://docs.headroomlabs.ai/docs/image-compression) ### Persistent Memory Hierarchical memory (user/session/agent/turn) with SQLite + HNSW backends. Survives across conversations. [Learn more →](https://docs.headroomlabs.ai/docs/memory) ### Failure Learning Reads past sessions, finds failed tool calls, correlates with what succeeded, writes learnings to CLAUDE.md. [Learn more →](https://docs.headroomlabs.ai/docs/failure-learning) ### Multi-Agent Context Compress what moves between agents. Any framework. ctx = SharedContext() ctx.put("research", big\_output) summary = ctx.get("research") [Learn more →](https://docs.headroomlabs.ai/docs/shared-context) ### Metrics & Observability Prometheus endpoint, per-request logging, cost tracking, budget limits, pipeline timing breakdowns. [Learn more →](https://docs.headroomlabs.ai/docs/metrics) [Framework Integrations](https://docs.headroomlabs.ai/docs#framework-integrations) ----------------------------------------------------------------------------------- ### LangChain Wrap any chat model. Supports memory, retrievers, tools, streaming, async. from headroom.integrations.langchain import HeadroomChatModel llm = HeadroomChatModel(ChatOpenAI()) [LangChain Guide →](https://docs.headroomlabs.ai/docs/langchain) ### Agno Full agent framework integration with observability hooks. from headroom.integrations.agno import HeadroomAgnoModel model = HeadroomAgnoModel(Claude()) agent = Agent(model=model) [Agno Guide →](https://docs.headroomlabs.ai/docs/agno) ### Strands Model wrapping + tool output hook provider for Strands Agents. from headroom.integrations.strands import HeadroomStrandsModel model = HeadroomStrandsModel(...) agent = Agent(model=model) [Strands Guide →](https://docs.headroomlabs.ai/docs/strands) ### MCP Tools Three tools for Claude Code, Cursor, or any MCP client: headroom\_compress, headroom\_retrieve, headroom\_stats. headroom mcp install && claude [MCP Tools Guide →](https://docs.headroomlabs.ai/docs/mcp) ### TypeScript SDK compress(), Vercel AI SDK middleware, OpenAI and Anthropic client wrappers. npm install headroom-ai [TypeScript SDK Guide →](https://docs.headroomlabs.ai/docs/vercel-ai-sdk) ### Vercel AI SDK One-liner withHeadroom() or headroomMiddleware() for any Vercel AI SDK model. import { withHeadroom } from 'headroom-ai/vercel-ai' const model = withHeadroom(openai('gpt-4o')) [Vercel AI SDK Guide →](https://docs.headroomlabs.ai/docs/vercel-ai-sdk) [All integration patterns →](https://docs.headroomlabs.ai/docs/quickstart) [Nothing is lost](https://docs.headroomlabs.ai/docs#nothing-is-lost) --------------------------------------------------------------------- Compressed content goes into the CCR store (Compress-Cache-Retrieve). The LLM gets a `headroom_retrieve` tool and can fetch full originals when it needs more detail. Compression is aggressive but reversible. [Next steps](https://docs.headroomlabs.ai/docs#next-steps) ----------------------------------------------------------- [### Quickstart](https://docs.headroomlabs.ai/docs/quickstart) [### Installation](https://docs.headroomlabs.ai/docs/installation) [### Proxy Server](https://docs.headroomlabs.ai/docs/proxy) [### Vercel AI SDK](https://docs.headroomlabs.ai/docs/vercel-ai-sdk) [### LangChain](https://docs.headroomlabs.ai/docs/langchain) [### How Compression Works](https://docs.headroomlabs.ai/docs/how-compression-works) [Quickstart\ \ Get Headroom running in 5 minutes. Install, compress, and send to your LLM with fewer tokens.](https://docs.headroomlabs.ai/docs/quickstart) ### On this page [Quick preview](https://docs.headroomlabs.ai/docs#quick-preview) [What gets compressed](https://docs.headroomlabs.ai/docs#what-gets-compressed) [Where Headroom fits](https://docs.headroomlabs.ai/docs#where-headroom-fits) [Real-world results](https://docs.headroomlabs.ai/docs#real-world-results) [Key Features](https://docs.headroomlabs.ai/docs#key-features) [Framework Integrations](https://docs.headroomlabs.ai/docs#framework-integrations) [Nothing is lost](https://docs.headroomlabs.ai/docs#nothing-is-lost) [Next steps](https://docs.headroomlabs.ai/docs#next-steps) --- # Agent Orchestration | Headroom Agent Orchestration =================== Keep repeated agent wakes cache-friendly while using CCR for lossless memory digest retrieval. Copy MarkdownOpen Repeated agent wakes usually rebuild the same expensive prompt shape. The useful split is simple: keep the byte-stable prefix first, keep the volatile wake-specific digest later, and keep run-specific context at the end so the repeated prefix stays cacheable. [Repeated wake anatomy](https://docs.headroomlabs.ai/docs/agent-orchestration#repeated-wake-anatomy) ----------------------------------------------------------------------------------------------------- Each wake tends to carry four different layers: * stable instructions, project rules, and tool contracts * the current task and other instruction-bearing fields * the volatile per-wake memory digest * recent loop state, tool output, and child-agent context The stable layers should stay byte-identical across wakes. The volatile layers should move to the live zone, where they can change without invalidating the cacheable prefix. [CacheAligner is detector-only](https://docs.headroomlabs.ai/docs/agent-orchestration#cachealigner-is-detector-only) --------------------------------------------------------------------------------------------------------------------- CacheAligner does not rewrite messages. It inspects the prefix, emits warnings for volatile content, and records observability data so callers can fix their own assembly logic. The detector reports: | Field | What it tells you | | --- | --- | | `warnings` | Which parts of the prefix look unstable | | `cache_metrics.stable_prefix_bytes` | Size of the stable prefix in bytes | | `cache_metrics.stable_prefix_tokens_est` | Estimated token size of the stable prefix | | `cache_metrics.stable_prefix_hash` | Hash of the stable prefix for repeated-wake comparison | | `cache_metrics.prefix_changed` | Whether the stable prefix drifted since the last wake | | `cache_metrics.previous_hash` | Hash from the previous wake, when available | | `markers` | The emitted `stable_prefix_hash` marker for downstream observability | If CacheAligner warns about drift, keep the prefix stable in the caller. The transform is a detector, not a repair pass. [Stable prefix layout](https://docs.headroomlabs.ai/docs/agent-orchestration#stable-prefix-layout) --------------------------------------------------------------------------------------------------- Put the byte-stable prefix first, then the live wake digest, then any run-specific context. That layout keeps the provider cache path predictable: 1. Stable instructions stay identical. 2. The wake digest changes without disturbing the cacheable prefix. 3. Recent tool output and child-agent context stay outside the repeatable prefix. [Real wake measurements](https://docs.headroomlabs.ai/docs/agent-orchestration#real-wake-measurements) ------------------------------------------------------------------------------------------------------- Use the issue comment guidance when measuring real wakes: | Field | Record | | --- | --- | | Prompt section id | Name the section that changed or repeated | | Byte/token estimate | Capture the section size before and after curation | | Digest version | Track which digest schema was used | | Cache-hit expectation | Note whether the section should stay cacheable | | Compressed size | Record the compacted payload size | | Section type | Mark the section as instruction-bearing or safe to compress | That checklist helps separate the repeated prefix from the volatile digest before you decide where CCR belongs. [CCR digest curation](https://docs.headroomlabs.ai/docs/agent-orchestration#ccr-digest-curation) ------------------------------------------------------------------------------------------------- CCR keeps compression reversible. The compressed digest can stay compact while the original backing detail remains recoverable from the local store. The pieces that matter here are: * `headroom_retrieve` for on-demand recovery of stored originals * `HEADROOM_CCR_TTL_SECONDS` for sizing the local store lifetime * `compression_strategy` as the authoritative discriminator on stored CCR entries For routing decisions, the same rule in plain terms is: headroom\_retrieve recovers originals, HEADROOM\_CCR\_TTL\_SECONDS sizes the local lifetime, compression\_strategy identifies the producing path, and shape inference is not the routing authority. When a stored original expires, regenerate the digest or re-read the source content. Do not infer routing from payload shape. Use the stored `compression_strategy` metadata to understand how the original was produced. [Digest routing](https://docs.headroomlabs.ai/docs/agent-orchestration#digest-routing) --------------------------------------------------------------------------------------- Route fields by how much exact wording they need at wake time. | Field | Suggested handling | Why | | --- | --- | --- | | Current task | Verbatim | It is instruction-bearing and changes the next action | | Hard constraints | Verbatim | These are safety and acceptance boundaries | | Definitions of done | Verbatim | The wording needs to survive every wake intact | | Irreversible decisions | Verbatim summary plus retrievable backing detail | The summary stays short, the backing detail stays exact | | Open threads | Compact summary plus CCR-backed backing detail | The live thread can shrink while the source stays recoverable | | Learnings | Compact summary | They inform the next wake without needing exact prose | | File/search/tool outputs | CCR-backed compression | These are large backing details that should stay retrievable | | Prose notes | Compact summary, CCR-backed when bulky | Keep the digest readable without losing source detail | | Bulk JSON-ish arrays | SmartCrusher or ContentRouter, with CCR-backed backing detail when needed | Structured blobs are usually the first thing to explode in size | The important boundary is simple: instruction-bearing fields stay verbatim, or they get a verbatim compact summary plus retrievable backing detail. CCR-backed content is the backing detail, not the instruction itself. Hard constraints stay verbatim; current task text stays verbatim; file/search/tool outputs use CCR-backed backing detail when they are large enough to compress. [Integration modes](https://docs.headroomlabs.ai/docs/agent-orchestration#integration-modes) --------------------------------------------------------------------------------------------- Choose the integration mode by where the orchestrator controls message assembly. | Mode | Use when | Notes | | --- | --- | --- | | Proxy | You want spawned agents and normal client traffic to pass through a local provider endpoint | Good for `headroom proxy --mode cache` and provider base URL routing | | Library | The orchestrator owns message assembly and wants to shape the digest before launch | Best when the caller can decide what becomes stable prefix versus live digest | | MCP | Agents need on-demand compression and retrieval tools | Best when `headroom_retrieve` should be available as a tool | | Proxy plus MCP | You need traffic shaping and tool-level retrieval together | Useful when both the provider edge and the agent toolset matter | [Local-first deployment](https://docs.headroomlabs.ai/docs/agent-orchestration#local-first-deployment) ------------------------------------------------------------------------------------------------------- Keep the deployment local to the user or session: * run one local proxy or MCP process per user session * do not assume a central proxy * treat the local CCR store as process-local unless the deployment explicitly shares it * size the TTL for the longest realistic autonomous run * assume a store can expire before the run finishes, then regenerate or re-read the source content That model keeps the data boundary obvious. The orchestrator can still recover backing detail, but the cacheable prefix stays small and stable. [Source-backed caveats](https://docs.headroomlabs.ai/docs/agent-orchestration#source-backed-caveats) ----------------------------------------------------------------------------------------------------- * Prompt-cache hits require a byte-identical stable prefix. * CacheAligner identifies drift, it does not repair prompt assembly. * CCR retrieval depends on store lifetime and stored hashes. * Provider-neutral guidance still applies, so keep the guide away from Orcha-specific runtime claims. * Use `compression_strategy` to read stored CCR intent, not payload shape. [Cache Optimization\ \ Stabilize message prefixes for provider KV cache hits and configure provider-specific caching strategies.](https://docs.headroomlabs.ai/docs/cache-optimization) [Context Management\ \ Automatic live-zone-only context management that compresses the newest content blocks while preserving the provider cache hot zone.](https://docs.headroomlabs.ai/docs/context-management) ### On this page [Repeated wake anatomy](https://docs.headroomlabs.ai/docs/agent-orchestration#repeated-wake-anatomy) [CacheAligner is detector-only](https://docs.headroomlabs.ai/docs/agent-orchestration#cachealigner-is-detector-only) [Stable prefix layout](https://docs.headroomlabs.ai/docs/agent-orchestration#stable-prefix-layout) [Real wake measurements](https://docs.headroomlabs.ai/docs/agent-orchestration#real-wake-measurements) [CCR digest curation](https://docs.headroomlabs.ai/docs/agent-orchestration#ccr-digest-curation) [Digest routing](https://docs.headroomlabs.ai/docs/agent-orchestration#digest-routing) [Integration modes](https://docs.headroomlabs.ai/docs/agent-orchestration#integration-modes) [Local-first deployment](https://docs.headroomlabs.ai/docs/agent-orchestration#local-first-deployment) [Source-backed caveats](https://docs.headroomlabs.ai/docs/agent-orchestration#source-backed-caveats) --- # Agno | Headroom Agno ==== Automatic context compression for Agno AI agents with model wrapping and observability hooks. Copy MarkdownOpen Headroom integrates with [Agno](https://github.com/agno-agi/agno) (formerly Phidata) to compress context for AI agents. Wrap any Agno model for automatic optimization, and use hooks for observability. [Installation](https://docs.headroomlabs.ai/docs/agno#installation) -------------------------------------------------------------------- pip install "headroom-ai[agno]" agno [Quick start](https://docs.headroomlabs.ai/docs/agno#quick-start) ------------------------------------------------------------------ from agno.agent import Agent from agno.models.openai import OpenAIChat from headroom.integrations.agno import HeadroomAgnoModel model = HeadroomAgnoModel(wrapped_model=OpenAIChat(id="gpt-4o")) agent = Agent(model=model) response = agent.run("What's the capital of France?") print(f"Tokens saved: {model.total_tokens_saved}") print(model.get_savings_summary()) # {'total_requests': 1, 'total_tokens_saved': 245, 'average_savings_percent': 12.3, # 'total_tokens_before': 1990, 'total_tokens_after': 1745} Works with any Agno provider: from agno.models.anthropic import Claude from agno.models.google import Gemini claude_model = HeadroomAgnoModel(wrapped_model=Claude(id="claude-sonnet-4-20250514")) gemini_model = HeadroomAgnoModel(wrapped_model=Gemini(id="gemini-2.0-flash")) [Observability hooks](https://docs.headroomlabs.ai/docs/agno#observability-hooks) ---------------------------------------------------------------------------------- Use hooks for detailed tracking without modifying your model: from headroom.integrations.agno import ( HeadroomAgnoModel, HeadroomPreHook, HeadroomPostHook, ) model = HeadroomAgnoModel(wrapped_model=OpenAIChat(id="gpt-4o")) pre_hook = HeadroomPreHook() post_hook = HeadroomPostHook(token_alert_threshold=10000) agent = Agent( model=model, pre_hooks=[pre_hook], post_hooks=[post_hook], ) response = agent.run("Analyze this large dataset...") # Check for alerts if post_hook.alerts: print(f"{len(post_hook.alerts)} requests exceeded threshold") Or use the convenience factory: from headroom.integrations.agno import create_headroom_hooks pre_hook, post_hook = create_headroom_hooks( token_alert_threshold=5000, log_level="DEBUG", ) [Tool-heavy agents](https://docs.headroomlabs.ai/docs/agno#tool-heavy-agents) ------------------------------------------------------------------------------ Tool outputs (JSON, logs, search results) see the biggest compression gains at 70-90% reduction: from agno.tools.duckduckgo import DuckDuckGoTools model = HeadroomAgnoModel(wrapped_model=OpenAIChat(id="gpt-4o")) agent = Agent( model=model, tools=[DuckDuckGoTools()], show_tool_calls=True, ) response = agent.run("Research the latest AI developments") print(f"Tokens saved: {model.total_tokens_saved}") [Async support](https://docs.headroomlabs.ai/docs/agno#async-support) ---------------------------------------------------------------------- import asyncio async def process(): model = HeadroomAgnoModel(wrapped_model=OpenAIChat(id="gpt-4o")) response = await model.aresponse(messages) async for chunk in model.aresponse_stream(messages): print(chunk, end="", flush=True) asyncio.run(process()) [Standalone message optimization](https://docs.headroomlabs.ai/docs/agno#standalone-message-optimization) ---------------------------------------------------------------------------------------------------------- Optimize messages without wrapping a model: from headroom.integrations.agno import optimize_messages optimized, metrics = optimize_messages(messages, model="gpt-4o") print(f"Tokens saved: {metrics['tokens_saved']}") [Session management](https://docs.headroomlabs.ai/docs/agno#session-management) -------------------------------------------------------------------------------- Reset metrics between sessions: model = HeadroomAgnoModel(wrapped_model=OpenAIChat(id="gpt-4o")) # Session 1 agent.run("First conversation...") print(model.get_savings_summary()) # Reset for new session model.reset() # Session 2 starts fresh agent.run("Second conversation...") [Supported providers](https://docs.headroomlabs.ai/docs/agno#supported-providers) ---------------------------------------------------------------------------------- | Provider | Agno Model | Auto-Detected | | --- | --- | --- | | OpenAI | `OpenAIChat`, `OpenAILike` | Yes | | Anthropic | `Claude`, `AwsBedrock` | Yes | | Google | `Gemini`, `VertexAI` | Yes | | Groq | `Groq` | Yes | | Mistral | `Mistral` | Yes | | Ollama | `Ollama` | Yes | [LangChain & LangGraph\ \ Compress context in LangChain chat models, memory, retrievers, and LangGraph agents.](https://docs.headroomlabs.ai/docs/langchain) [Strands\ \ Context compression for Strands Agents via model wrapping and hook-based tool output compression.](https://docs.headroomlabs.ai/docs/strands) ### On this page [Installation](https://docs.headroomlabs.ai/docs/agno#installation) [Quick start](https://docs.headroomlabs.ai/docs/agno#quick-start) [Observability hooks](https://docs.headroomlabs.ai/docs/agno#observability-hooks) [Tool-heavy agents](https://docs.headroomlabs.ai/docs/agno#tool-heavy-agents) [Async support](https://docs.headroomlabs.ai/docs/agno#async-support) [Standalone message optimization](https://docs.headroomlabs.ai/docs/agno#standalone-message-optimization) [Session management](https://docs.headroomlabs.ai/docs/agno#session-management) [Supported providers](https://docs.headroomlabs.ai/docs/agno#supported-providers) --- # Anthropic SDK | Headroom Anthropic SDK ============= Auto-compress messages in the Anthropic TypeScript SDK with a single withHeadroom() wrapper. Copy MarkdownOpen Headroom wraps the Anthropic TypeScript SDK to automatically compress messages before every `messages.create()` call. All other methods pass through unchanged. [Installation](https://docs.headroomlabs.ai/docs/anthropic-sdk#installation) ----------------------------------------------------------------------------- npm install headroom-ai @anthropic-ai/sdk Proxy required The TypeScript SDK sends messages to a local Headroom proxy for compression. Start the proxy before using the SDK: pip install "headroom-ai[proxy]" headroom proxy [Quick start](https://docs.headroomlabs.ai/docs/anthropic-sdk#quick-start) --------------------------------------------------------------------------- import { withHeadroom } from 'headroom-ai/anthropic'; import Anthropic from '@anthropic-ai/sdk'; const client = withHeadroom(new Anthropic()); const response = await client.messages.create({ model: 'claude-sonnet-4-5-20250929', messages: longConversation, max_tokens: 1024, }); Every call to `client.messages.create()` compresses messages first. The response format is identical to the unwrapped client. [How it works](https://docs.headroomlabs.ai/docs/anthropic-sdk#how-it-works) ----------------------------------------------------------------------------- `withHeadroom()` returns a proxy around your Anthropic client that intercepts `messages.create()`: 1. Converts Anthropic-format messages to OpenAI format 2. Sends them to the Headroom proxy's `/v1/compress` endpoint 3. Converts the compressed messages back to Anthropic format 4. Forwards the request to Anthropic as normal ### [Message format conversion](https://docs.headroomlabs.ai/docs/anthropic-sdk#message-format-conversion) The adapter handles the full Anthropic message format including content blocks: | Anthropic format | OpenAI format | | --- | --- | | `{ type: "text", text: "..." }` | `{ role: "user", content: "..." }` | | `{ type: "tool_use", id, name, input }` | `{ tool_calls: [{ id, function: { name, arguments } }] }` | | `{ type: "tool_result", tool_use_id, content }` | `{ role: "tool", tool_call_id, content }` | This conversion is lossless. Your request and response behave identically to an unwrapped client. The conversion is an SDK choice, not an API requirement `POST /v1/compress` does no format conversion and compresses Anthropic content blocks natively — see [Message format](https://docs.headroomlabs.ai/docs/proxy#message-format) . If you are calling the endpoint directly (from a gateway, or LiteLLM's `headroom` guardrail), send Anthropic-shaped messages as-is; you get the same shape back. Only this TypeScript adapter converts, because it normalises on OpenAI types internally. [Options](https://docs.headroomlabs.ai/docs/anthropic-sdk#options) ------------------------------------------------------------------- Pass compression options as the second argument: import { withHeadroom } from 'headroom-ai/anthropic'; import Anthropic from '@anthropic-ai/sdk'; const client = withHeadroom(new Anthropic(), { model: 'claude-sonnet-4-5-20250929', baseUrl: 'http://localhost:8787', }); [Streaming](https://docs.headroomlabs.ai/docs/anthropic-sdk#streaming) ----------------------------------------------------------------------- Streaming works normally. Compression happens before the request: import { withHeadroom } from 'headroom-ai/anthropic'; import Anthropic from '@anthropic-ai/sdk'; const client = withHeadroom(new Anthropic()); const stream = await client.messages.create({ model: 'claude-sonnet-4-5-20250929', messages: longConversation, max_tokens: 1024, stream: true, }); [Tool use](https://docs.headroomlabs.ai/docs/anthropic-sdk#tool-use) --------------------------------------------------------------------- Tool results are where compression has the biggest impact. Large JSON payloads from tool calls are compressed automatically: import { withHeadroom } from 'headroom-ai/anthropic'; import Anthropic from '@anthropic-ai/sdk'; const client = withHeadroom(new Anthropic()); const response = await client.messages.create({ model: 'claude-sonnet-4-5-20250929', max_tokens: 1024, messages: [\ { role: 'user', content: 'What went wrong?' },\ {\ role: 'assistant',\ content: [\ { type: 'tool_use', id: 'toolu_1', name: 'get_logs', input: { service: 'api' } },\ ],\ },\ {\ role: 'user',\ content: [\ {\ type: 'tool_result',\ tool_use_id: 'toolu_1',\ content: hugeLogOutput, // Compressed automatically\ },\ ],\ },\ ], tools: [{ name: 'get_logs', description: 'Get logs', input_schema: { type: 'object', properties: {} } }], }); [OpenAI SDK\ \ Auto-compress messages in the OpenAI Node.js SDK with a single withHeadroom() wrapper.](https://docs.headroomlabs.ai/docs/openai-sdk) [LangChain & LangGraph\ \ Compress context in LangChain chat models, memory, retrievers, and LangGraph agents.](https://docs.headroomlabs.ai/docs/langchain) ### On this page [Installation](https://docs.headroomlabs.ai/docs/anthropic-sdk#installation) [Quick start](https://docs.headroomlabs.ai/docs/anthropic-sdk#quick-start) [How it works](https://docs.headroomlabs.ai/docs/anthropic-sdk#how-it-works) [Message format conversion](https://docs.headroomlabs.ai/docs/anthropic-sdk#message-format-conversion) [Options](https://docs.headroomlabs.ai/docs/anthropic-sdk#options) [Streaming](https://docs.headroomlabs.ai/docs/anthropic-sdk#streaming) [Tool use](https://docs.headroomlabs.ai/docs/anthropic-sdk#tool-use) --- # API Reference | Headroom API Reference ============= Complete API reference for the Headroom Python and TypeScript SDKs. Core client, configuration types, result types, errors, and utilities. Copy MarkdownOpen Complete API reference for the Headroom Python and TypeScript SDKs. [Core](https://docs.headroomlabs.ai/docs/api-reference#core) ------------------------------------------------------------- ### [HeadroomClient](https://docs.headroomlabs.ai/docs/api-reference#headroomclient) The main entry point for the Headroom SDK. TypeScriptPython Prop Type `baseUrl?`string `apiKey?`string `timeout?`number `fallback?`boolean `retries?`number import { HeadroomClient } from 'headroom-ai'; const client = new HeadroomClient({ baseUrl: 'http://localhost:8787', apiKey: 'your-api-key', timeout: 30_000, fallback: true, retries: 2, }); **Constructor Parameters** Prop Type `original_client?`OpenAI | Anthropic `provider?`Provider `default_mode?`"audit" | "optimize" `store_url?`str | None `config?`HeadroomConfig | None `enable_cache_optimizer?`bool `enable_semantic_cache?`bool `model_context_limits?`dict\[str, int\] from headroom import HeadroomClient, OpenAIProvider, HeadroomConfig, SmartCrusherConfig, CacheAlignerConfig from openai import OpenAI client = HeadroomClient( original_client=OpenAI(), provider=OpenAIProvider(), default_mode="optimize", config=HeadroomConfig( smart_crusher=SmartCrusherConfig(min_tokens_to_crush=100), cache_aligner=CacheAlignerConfig(enabled=True), ), ) ### [chat.completions.create()](https://docs.headroomlabs.ai/docs/api-reference#chatcompletionscreate) Create a chat completion with optional optimization. TypeScriptPython The TypeScript SDK uses `compress()` to optimize messages before sending them to your LLM client: import { compress } from 'headroom-ai'; const result = await compress(messages, { model: 'gpt-4o', tokenBudget: 100_000, }); // Then pass result.messages to your LLM client Accepts all standard OpenAI/Anthropic parameters plus Headroom-specific overrides: Prop Type `headroom_mode?`"audit" | "optimize" | "simulate" `headroom_query?`str `headroom_output_buffer_tokens?`int `headroom_keep_turns?`int response = client.chat.completions.create( model="gpt-4o", messages=[...], headroom_mode="optimize", headroom_keep_turns=5, ) headroom\_tool\_profiles has no effect in 0.37.0 `headroom_tool_profiles` is accepted but not currently wired through `TransformPipeline.apply()`, so passing it changes nothing. To keep specific tool outputs from being compressed as aggressively, use `config.smart_crusher.max_items_after_crush` or `config.smart_crusher.enabled = False` instead — see [Compression Too Aggressive](https://docs.headroomlabs.ai/docs/troubleshooting#compression-too-aggressive) . ### [chat.completions.simulate()](https://docs.headroomlabs.ai/docs/api-reference#chatcompletionssimulate) Preview optimization without making an API call. plan = client.chat.completions.simulate( model="gpt-4o", messages=[...], ) print(f"Tokens: {plan.tokens_before} -> {plan.tokens_after}") print(f"Savings: {plan.tokens_saved/plan.tokens_before*100:.1f}%") print(f"Transforms: {plan.transforms}") **Returns:** `SimulationResult` ### [compress() (TypeScript)](https://docs.headroomlabs.ai/docs/api-reference#compress-typescript) Top-level function to compress messages via the Headroom proxy. Prop Type `model?`string `baseUrl?`string `apiKey?`string `timeout?`number `fallback?`boolean `retries?`number `client?`HeadroomClientInterface `tokenBudget?`number `hooks?`CompressionHooks import { compress } from 'headroom-ai'; const result = await compress(messages, { model: 'gpt-4o', baseUrl: 'http://localhost:8787', timeout: 15_000, fallback: true, retries: 2, tokenBudget: 100_000, }); ### [get\_stats()](https://docs.headroomlabs.ai/docs/api-reference#get_stats) Quick stats for the current session (no database query). stats = client.get_stats() # Returns dict with "session", "config", and "transforms" keys ### [get\_metrics()](https://docs.headroomlabs.ai/docs/api-reference#get_metrics) Query stored metrics from the database. from datetime import datetime, timedelta metrics = client.get_metrics( start_time=datetime.utcnow() - timedelta(hours=1), limit=100, ) ### [get\_summary()](https://docs.headroomlabs.ai/docs/api-reference#get_summary) Aggregate statistics across all stored metrics. summary = client.get_summary() # Returns dict with total_requests, total_tokens_before, total_tokens_after, # total_tokens_saved, avg_tokens_saved, avg_cache_alignment, # audit_count, optimize_count ### [validate\_setup()](https://docs.headroomlabs.ai/docs/api-reference#validate_setup) Validate that the client is configured correctly. result = client.validate_setup() # Returns {"valid": bool, "provider": {...}, "storage": {...}, # "config": {...}, "cache_optimizer": {...}}, each with "ok"/"error" if not result["valid"]: for key in ("provider", "storage", "config", "cache_optimizer"): if not result[key]["ok"]: print(f" - {key}: {result[key]['error']}") * * * [Configuration](https://docs.headroomlabs.ai/docs/api-reference#configuration) ------------------------------------------------------------------------------- ### [SmartCrusherConfig](https://docs.headroomlabs.ai/docs/api-reference#smartcrusherconfig) TypeScriptPython Prop Type `enabled?`boolean `minItemsToAnalyze?`number `minTokensToCrush?`number `varianceThreshold?`number `uniquenessThreshold?`number `similarityThreshold?`number `maxItemsAfterCrush?`number `preserveChangePoints?`boolean `useFeedbackHints?`boolean `toinConfidenceThreshold?`number `relevance?`RelevanceScorerConfig `anchor?`AnchorConfig `dedupIdenticalItems?`boolean `firstFraction?`number `lastFraction?`number Prop Type `min_tokens_to_crush?`int `min_items_to_analyze?`int `max_items_after_crush?`int `variance_threshold?`float `uniqueness_threshold?`float `similarity_threshold?`float `preserve_change_points?`bool `use_feedback_hints?`bool `dedup_identical_items?`bool `first_fraction?`float `last_fraction?`float from headroom import SmartCrusherConfig config = SmartCrusherConfig( min_tokens_to_crush=200, max_items_after_crush=15, variance_threshold=2.0, preserve_change_points=True, ) ### [CacheAlignerConfig](https://docs.headroomlabs.ai/docs/api-reference#cachealignerconfig) TypeScriptPython Prop Type `enabled?`boolean `useDynamicDetector?`boolean `detectionTiers?`("regex" | "ner" | "semantic")\[\] `extraDynamicLabels?`string\[\] `entropyThreshold?`number `datePatterns?`string\[\] `normalizeWhitespace?`boolean `collapseBlankLines?`boolean `dynamicTailSeparator?`string Prop Type `enabled?`bool `use_dynamic_detector?`bool `detection_tiers?`list\[Literal\['regex', 'ner', 'semantic'\]\] `extra_dynamic_labels?`list\[str\] `entropy_threshold?`float `date_patterns?`list\[str\] `normalize_whitespace?`bool `collapse_blank_lines?`bool `dynamic_tail_separator?`str from headroom import CacheAlignerConfig config = CacheAlignerConfig( enabled=True, use_dynamic_detector=True, normalize_whitespace=True, entropy_threshold=0.7, ) ### [Context management](https://docs.headroomlabs.ai/docs/api-reference#context-management) Context management is now handled automatically inside the pipeline (live-zone-only compression). Headroom never drops messages from the conversation history; it compresses only the newest content blocks (latest user message, latest tool result) and keeps the cache hot zone — system prompt, tools, and older turns — untouched. Use the `headroom_keep_turns` / `headroom_output_buffer_tokens` per-request overrides to tune behavior. The `RollingWindowConfig`, `IntelligentContextConfig`, and `ScoringWeights` classes have been retired from the Python package (`from headroom import RollingWindowConfig` fails) and from the compression pipeline itself; the TypeScript SDK's `HeadroomConfig` type still exports the equivalent field names as unused legacy type surface, wired to nothing. ### [HeadroomConfig](https://docs.headroomlabs.ai/docs/api-reference#headroomconfig) TypeScriptPython Prop Type `storeUrl?`string `defaultMode?`HeadroomMode `modelContextLimits?`Record `smartCrusher?`SmartCrusherConfig `cacheAligner?`CacheAlignerConfig `cacheOptimizer?`CacheOptimizerConfig `ccr?`CCRConfig `prefixFreeze?`PrefixFreezeConfig `contentRouterEnabled?`boolean `generateDiffArtifact?`boolean The top-level config object that contains all sub-configurations: from headroom import HeadroomConfig config = HeadroomConfig() config.smart_crusher.min_tokens_to_crush = 100 config.cache_aligner.enabled = True # Note: rolling_window has been removed — use headroom_keep_turns per-request instead ### [RelevanceScorerConfig](https://docs.headroomlabs.ai/docs/api-reference#relevancescorerconfig) TypeScriptPython Prop Type `tier?`RelevanceTier `bm25K1?`number `bm25B?`number `embeddingModel?`string `hybridAlpha?`number `adaptiveAlpha?`boolean `relevanceThreshold?`number Prop Type `tier?`"bm25" | "embedding" | "hybrid" `bm25_k1?`float `bm25_b?`float `embedding_model?`str `hybrid_alpha?`float `adaptive_alpha?`bool `relevance_threshold?`float * * * [Results](https://docs.headroomlabs.ai/docs/api-reference#results) ------------------------------------------------------------------- ### [CompressResult (TypeScript)](https://docs.headroomlabs.ai/docs/api-reference#compressresult-typescript) Prop Type `messages?`any\[\] `tokensBefore?`number `tokensAfter?`number `tokensSaved?`number `compressionRatio?`number `transformsApplied?`string\[\] `ccrHashes?`string\[\] `compressed?`boolean ### [SimulationResult (Python)](https://docs.headroomlabs.ai/docs/api-reference#simulationresult-python) Prop Type `tokens_before?`int `tokens_after?`int `tokens_saved?`int `transforms?`list\[str\] `waste_signals?`dict\[str, int\] ### [WasteSignals (Python)](https://docs.headroomlabs.ai/docs/api-reference#wastesignals-python) `plan.waste_signals` is a plain `dict[str, int]`, not a class instance: Prop Type `json_bloat?`int `html_noise?`int `base64?`int `whitespace?`int `dynamic_date?`int `repetition?`int `reread?`int `reread_compressed?`int ### [RequestMetrics (Python)](https://docs.headroomlabs.ai/docs/api-reference#requestmetrics-python) Prop Type `request_id?`str `timestamp?`datetime `model?`str `tokens_input_before?`int `tokens_input_after?`int `tokens_output?`int `transforms_applied?`list\[str\] Cost estimates are computed on demand inside `HeadroomClient` (`estimate_cost()` on the provider) rather than stored on `RequestMetrics`. * * * [Providers](https://docs.headroomlabs.ai/docs/api-reference#providers) ----------------------------------------------------------------------- ### [OpenAIProvider](https://docs.headroomlabs.ai/docs/api-reference#openaiprovider) from headroom import OpenAIProvider provider = OpenAIProvider() counter = provider.get_token_counter("gpt-4o") tokens = counter.count_text("Hello, world!") limit = provider.get_context_limit("gpt-4o") # 128000 cost = provider.estimate_cost(input_tokens=1000, output_tokens=500, model="gpt-4o") ### [AnthropicProvider](https://docs.headroomlabs.ai/docs/api-reference#anthropicprovider) from headroom import AnthropicProvider from anthropic import Anthropic provider = AnthropicProvider( client=Anthropic(), ) counter = provider.get_token_counter("claude-3-5-sonnet-latest") tokens = counter.count_messages(messages) # Accurate count via API ### [GoogleProvider](https://docs.headroomlabs.ai/docs/api-reference#googleprovider) from headroom.providers import GoogleProvider provider = GoogleProvider() * * * [Relevance Scoring](https://docs.headroomlabs.ai/docs/api-reference#relevance-scoring) --------------------------------------------------------------------------------------- ### [create\_scorer()](https://docs.headroomlabs.ai/docs/api-reference#create_scorer) Factory function to create scorers: from headroom import create_scorer # Auto-select best available scorer scorer = create_scorer() # Explicitly choose type scorer = create_scorer(tier="hybrid", alpha=0.7) ### [BM25Scorer](https://docs.headroomlabs.ai/docs/api-reference#bm25scorer) Fast keyword-based scoring (zero dependencies): from headroom import BM25Scorer scorer = BM25Scorer() scores = scorer.score_batch(["item 1", "item 2"], "search query") ### [EmbeddingScorer](https://docs.headroomlabs.ai/docs/api-reference#embeddingscorer) Semantic similarity scoring (requires `headroom-ai[relevance]`): from headroom import EmbeddingScorer, embedding_available if embedding_available(): scorer = EmbeddingScorer(model_name="BAAI/bge-small-en-v1.5") scores = scorer.score_batch(items, query) ### [HybridScorer](https://docs.headroomlabs.ai/docs/api-reference#hybridscorer) Combines BM25 and embeddings: from headroom import HybridScorer scorer = HybridScorer(alpha=0.5) # 50% BM25, 50% embedding scores = scorer.score_batch(items, query) * * * [Transforms (Direct Use)](https://docs.headroomlabs.ai/docs/api-reference#transforms-direct-use) ------------------------------------------------------------------------------------------------- ### [SmartCrusher](https://docs.headroomlabs.ai/docs/api-reference#smartcrusher) from headroom import SmartCrusher import json crusher = SmartCrusher() result = crusher.crush(content=json.dumps({"results": [...]}), query="user query") ### [CacheAligner](https://docs.headroomlabs.ai/docs/api-reference#cachealigner) from headroom import CacheAligner, Tokenizer, OpenAIProvider provider = OpenAIProvider() tokenizer = Tokenizer(provider.get_token_counter("gpt-4o"), "gpt-4o") aligner = CacheAligner() result = aligner.apply(messages, tokenizer) ### [TransformPipeline](https://docs.headroomlabs.ai/docs/api-reference#transformpipeline) from headroom import TransformPipeline, SmartCrusher, CacheAligner pipeline = TransformPipeline(transforms=[\ SmartCrusher(),\ CacheAligner(),\ ]) # model_limit is required for direct pipeline use — HeadroomClient # supplies it automatically from the provider's context limits result = pipeline.apply(messages, "gpt-4o", model_limit=128_000) * * * [Errors](https://docs.headroomlabs.ai/docs/api-reference#errors) ----------------------------------------------------------------- TypeScriptPython | Exception | Meaning | | --- | --- | | `HeadroomError` | Base class for all errors | | `HeadroomConnectionError` | Cannot reach proxy | | `HeadroomAuthError` | 401 from proxy | | `HeadroomCompressError` | Compression failed (includes `statusCode`, `errorType`) | | `ConfigurationError` | Invalid configuration | | `ProviderError` | Provider issues | | `StorageError` | Storage failures | | `TokenizationError` | Token counting failed | | `CacheError` | Cache operations failed | | `ValidationError` | Validation failures | | `TransformError` | Transform execution failed | Use `mapProxyError(status, type, message)` to convert proxy error responses to the correct class. | Exception | Meaning | | --- | --- | | `HeadroomError` | Base class for all Headroom errors | | `ConfigurationError` | Invalid config values | | `ProviderError` | Provider issue (unknown model, etc.) | | `StorageError` | Database issue | | `CompressionError` | Compression failed (rare) | | `ValidationError` | Setup validation failed | All exceptions include a `details` dict with additional context. * * * [Utilities](https://docs.headroomlabs.ai/docs/api-reference#utilities) ----------------------------------------------------------------------- ### [Tokenizer](https://docs.headroomlabs.ai/docs/api-reference#tokenizer) from headroom import Tokenizer, count_tokens_text, count_tokens_messages, OpenAIProvider provider = OpenAIProvider() token_counter = provider.get_token_counter("gpt-4o") # Quick counting tokens = count_tokens_text("Hello, world!", token_counter) # With tokenizer instance tokenizer = Tokenizer(token_counter, "gpt-4o") tokens = tokenizer.count_text("Hello") tokens = tokenizer.count_messages(messages) ### [generate\_report()](https://docs.headroomlabs.ai/docs/api-reference#generate_report) Generate HTML/Markdown reports from stored metrics: from headroom import generate_report report = generate_report( store_url="sqlite:///headroom.db", format="html", period="day", ) * * * [TypeScript Message Types](https://docs.headroomlabs.ai/docs/api-reference#typescript-message-types) ----------------------------------------------------------------------------------------------------- Prop Type `role?`"system" | "user" | "assistant" | "tool" `content?`string | ContentPart\[\] | null `tool_calls?`ToolCall\[\] `tool_call_id?`string The TypeScript SDK uses the standard OpenAI message format with `SystemMessage`, `UserMessage`, `AssistantMessage`, and `ToolMessage` variants. [Simulation\ \ Preview compression results without making an LLM call. Use simulation for cost estimation, debugging, and understanding waste signals.](https://docs.headroomlabs.ai/docs/simulation) [Architecture\ \ How Headroom compresses LLM traffic — from request interception through the ContentRouter compression pipeline to provider cache optimization.](https://docs.headroomlabs.ai/docs/architecture) ### On this page [Core](https://docs.headroomlabs.ai/docs/api-reference#core) [HeadroomClient](https://docs.headroomlabs.ai/docs/api-reference#headroomclient) [chat.completions.create()](https://docs.headroomlabs.ai/docs/api-reference#chatcompletionscreate) [chat.completions.simulate()](https://docs.headroomlabs.ai/docs/api-reference#chatcompletionssimulate) [compress() (TypeScript)](https://docs.headroomlabs.ai/docs/api-reference#compress-typescript) [get\_stats()](https://docs.headroomlabs.ai/docs/api-reference#get_stats) [get\_metrics()](https://docs.headroomlabs.ai/docs/api-reference#get_metrics) [get\_summary()](https://docs.headroomlabs.ai/docs/api-reference#get_summary) [validate\_setup()](https://docs.headroomlabs.ai/docs/api-reference#validate_setup) [Configuration](https://docs.headroomlabs.ai/docs/api-reference#configuration) [SmartCrusherConfig](https://docs.headroomlabs.ai/docs/api-reference#smartcrusherconfig) [CacheAlignerConfig](https://docs.headroomlabs.ai/docs/api-reference#cachealignerconfig) [Context management](https://docs.headroomlabs.ai/docs/api-reference#context-management) [HeadroomConfig](https://docs.headroomlabs.ai/docs/api-reference#headroomconfig) [RelevanceScorerConfig](https://docs.headroomlabs.ai/docs/api-reference#relevancescorerconfig) [Results](https://docs.headroomlabs.ai/docs/api-reference#results) [CompressResult (TypeScript)](https://docs.headroomlabs.ai/docs/api-reference#compressresult-typescript) [SimulationResult (Python)](https://docs.headroomlabs.ai/docs/api-reference#simulationresult-python) [WasteSignals (Python)](https://docs.headroomlabs.ai/docs/api-reference#wastesignals-python) [RequestMetrics (Python)](https://docs.headroomlabs.ai/docs/api-reference#requestmetrics-python) [Providers](https://docs.headroomlabs.ai/docs/api-reference#providers) [OpenAIProvider](https://docs.headroomlabs.ai/docs/api-reference#openaiprovider) [AnthropicProvider](https://docs.headroomlabs.ai/docs/api-reference#anthropicprovider) [GoogleProvider](https://docs.headroomlabs.ai/docs/api-reference#googleprovider) [Relevance Scoring](https://docs.headroomlabs.ai/docs/api-reference#relevance-scoring) [create\_scorer()](https://docs.headroomlabs.ai/docs/api-reference#create_scorer) [BM25Scorer](https://docs.headroomlabs.ai/docs/api-reference#bm25scorer) [EmbeddingScorer](https://docs.headroomlabs.ai/docs/api-reference#embeddingscorer) [HybridScorer](https://docs.headroomlabs.ai/docs/api-reference#hybridscorer) [Transforms (Direct Use)](https://docs.headroomlabs.ai/docs/api-reference#transforms-direct-use) [SmartCrusher](https://docs.headroomlabs.ai/docs/api-reference#smartcrusher) [CacheAligner](https://docs.headroomlabs.ai/docs/api-reference#cachealigner) [TransformPipeline](https://docs.headroomlabs.ai/docs/api-reference#transformpipeline) [Errors](https://docs.headroomlabs.ai/docs/api-reference#errors) [Utilities](https://docs.headroomlabs.ai/docs/api-reference#utilities) [Tokenizer](https://docs.headroomlabs.ai/docs/api-reference#tokenizer) [generate\_report()](https://docs.headroomlabs.ai/docs/api-reference#generate_report) [TypeScript Message Types](https://docs.headroomlabs.ai/docs/api-reference#typescript-message-types) --- # AutoGen | Headroom AutoGen ======= Automatic tool output compression for AutoGen agents with per-tool metrics tracking. Copy MarkdownOpen Headroom integrates with [AutoGen](https://github.com/microsoft/autogen) (`autogen-agentchat` >=0.7) to compress tool outputs before they enter the agent's model context. Tool-heavy agents that return large JSON arrays, database results, or verbose logs see the largest token reductions. [Installation](https://docs.headroomlabs.ai/docs/autogen#installation) ----------------------------------------------------------------------- pip install headroom-ai autogen-agentchat [Quick start](https://docs.headroomlabs.ai/docs/autogen#quick-start) --------------------------------------------------------------------- Wrap tools in one line: from autogen_agentchat.agents import AssistantAgent from autogen_core.tools import FunctionTool from headroom.integrations.autogen import wrap_tools_with_headroom def search_database(query: str) -> str: """Search the database and return results.""" return json.dumps({"results": [...], "total": 1000}) tool = FunctionTool(search_database, description="Search the database") wrapped = wrap_tools_with_headroom([tool]) agent = AssistantAgent( name="researcher", model_client=model_client, tools=wrapped, ) [Per-tool metrics](https://docs.headroomlabs.ai/docs/autogen#per-tool-metrics) ------------------------------------------------------------------------------- Track compression stats across all tool invocations: from headroom.integrations.autogen import get_tool_metrics metrics = get_tool_metrics() print(metrics.get_summary()) # { # 'total_invocations': 25, # 'total_compressions': 18, # 'total_chars_saved': 450000, # 'average_compression_ratio': 0.35, # 'by_tool': { # 'search_database': {'invocations': 15, 'compressions': 12, 'chars_saved': 320000}, # } # } Reset between sessions: from headroom.integrations.autogen import reset_tool_metrics reset_tool_metrics() [Custom configuration](https://docs.headroomlabs.ai/docs/autogen#custom-configuration) --------------------------------------------------------------------------------------- Control the compression threshold: wrapped = wrap_tools_with_headroom( [search_tool, log_tool], min_chars_to_compress=500, # Default: 1000 ) Use a dedicated metrics collector: from headroom.integrations.autogen import ToolMetricsCollector, wrap_tools_with_headroom collector = ToolMetricsCollector() wrapped = wrap_tools_with_headroom( [search_tool], metrics_collector=collector, ) print(collector.get_summary()) [Wrapping individual tools](https://docs.headroomlabs.ai/docs/autogen#wrapping-individual-tools) ------------------------------------------------------------------------------------------------- For finer control, wrap tools individually: from headroom.integrations.autogen import HeadroomToolWrapper wrapper = HeadroomToolWrapper( search_tool, min_chars_to_compress=500, ) # Get the wrapped FunctionTool compressed_tool = wrapper.as_function_tool() agent = AssistantAgent( name="researcher", model_client=model_client, tools=[compressed_tool], ) [Async support](https://docs.headroomlabs.ai/docs/autogen#async-support) ------------------------------------------------------------------------- AutoGen tools are natively async. The wrapper handles both sync and async tool functions transparently: async def async_search(query: str) -> str: """Async database search.""" results = await db.search(query) return json.dumps(results) tool = FunctionTool(async_search, description="Async search") wrapped = wrap_tools_with_headroom([tool]) # Compression works identically for async tools [How it works](https://docs.headroomlabs.ai/docs/autogen#how-it-works) ----------------------------------------------------------------------- AutoGen routes tool execution through `FunctionTool`, which wraps a plain Python function. The function's return value is stringified and becomes `FunctionExecutionResult.content` — what the LLM reads on its next turn. `HeadroomToolWrapper` creates a new `FunctionTool` with a wrapper function that: 1. Calls the original function 2. Checks if the stringified output exceeds `min_chars_to_compress` 3. If so, compresses via Headroom's `compress_tool_result()` 4. Records metrics and returns the compressed string The wrapper preserves the original tool's name, description, and parameter schema, so it works as a drop-in replacement. ### [Why not `tool_call_summary_formatter`?](https://docs.headroomlabs.ai/docs/autogen#why-not-tool_call_summary_formatter) AutoGen's `AssistantAgent` accepts a `tool_call_summary_formatter` parameter, which looks like a natural hook. However, it only controls the **final summary message** emitted after the tool loop exits — it does not touch the raw `FunctionExecutionResult` that gets added to `model_context` (what the LLM actually reads). Wrapping the function is the only clean interception point. [CrewAI\ \ Automatic tool output compression for CrewAI agents with per-tool metrics tracking.](https://docs.headroomlabs.ai/docs/crewai) [LiteLLM\ \ Add Headroom compression to LiteLLM with a single callback. Works with all 100+ supported providers.](https://docs.headroomlabs.ai/docs/litellm) ### On this page [Installation](https://docs.headroomlabs.ai/docs/autogen#installation) [Quick start](https://docs.headroomlabs.ai/docs/autogen#quick-start) [Per-tool metrics](https://docs.headroomlabs.ai/docs/autogen#per-tool-metrics) [Custom configuration](https://docs.headroomlabs.ai/docs/autogen#custom-configuration) [Wrapping individual tools](https://docs.headroomlabs.ai/docs/autogen#wrapping-individual-tools) [Async support](https://docs.headroomlabs.ai/docs/autogen#async-support) [How it works](https://docs.headroomlabs.ai/docs/autogen#how-it-works) [Why not `tool_call_summary_formatter`?](https://docs.headroomlabs.ai/docs/autogen#why-not-tool_call_summary_formatter) --- # Architecture | Headroom Architecture ============ How Headroom compresses LLM traffic — from request interception through the ContentRouter compression pipeline to provider cache optimization. Copy MarkdownOpen Headroom sits between your application and the LLM provider. It intercepts the request, compresses the parts that carry the most redundant tokens — tool outputs, file reads, logs, search results — and forwards the optimized request upstream. The provider's response is returned unchanged. [High-level flow](https://docs.headroomlabs.ai/docs/architecture#high-level-flow) ---------------------------------------------------------------------------------- +---------------------------------------------------------------+ | YOUR APPLICATION | +---------------------------------------------------------------+ | v +---------------------------------------------------------------+ | HEADROOM | | Proxy (FastAPI) · Python compress() · TS compress() | | | | | v | | Transform pipeline ──▶ ContentRouter | | (detect content type, route to one compressor) | | | | | v | | Backend (direct · LiteLLM · any-llm) | +---------------------------------------------------------------+ | v +---------------------------------------------------------------+ | OPENAI · ANTHROPIC · GOOGLE · BEDROCK · 100+ | +---------------------------------------------------------------+ [Entry points](https://docs.headroomlabs.ai/docs/architecture#entry-points) ---------------------------------------------------------------------------- Headroom can be used three ways, all feeding the same compression pipeline: | Entry point | How it works | Code changes | | --- | --- | --- | | **Proxy mode** | Run `headroom proxy` and point your client's base URL at it | Zero — just change the base URL | | **SDK mode** | Call `compress()` (Python or TypeScript) on your messages before you send them, or wrap an existing client with `HeadroomClient` for automatic per-call optimization and stats | Minimal — one function call, or a client wrapper | | **Integrations** | LangChain, Vercel AI SDK, Agno, Strands, LiteLLM, MCP adapters | Framework-specific setup | In proxy mode the server is a FastAPI app with per-provider handlers (Anthropic, OpenAI, Gemini, Bedrock) that each run the same compression pipeline before forwarding through the selected [backend](https://docs.headroomlabs.ai/docs/proxy#cloud-providers) . [The compression pipeline](https://docs.headroomlabs.ai/docs/architecture#the-compression-pipeline) ---------------------------------------------------------------------------------------------------- The proxy assembles a small, ordered pipeline. Every transform is independent, safe to skip, and **fails open** — on any error it returns the content unchanged and the request still goes through. 1. **Tool-result interceptor** _(canary opt-in)_ — light structural interceptors such as ast-grep Read outlining. Requires `HEADROOM_ROLLOUT_CHANNEL=canary` plus `--intercept-tool-results`. 2. **CacheAligner** _(off by default)_ — a detector that reports dynamic-prefix drift (dates, UUIDs, session tokens). It **never mutates, moves, or rewrites** content. It is disabled by default and hard-disabled inside the proxy; it exists to surface prefix-stability metrics, not to change your messages. 3. **ContentRouter** — the workhorse that does essentially all of the compression. See below. The pipeline **never drops or reorders messages**. Earlier versions shipped a "context manager" stage (rolling-window / intelligent-context scoring) that deleted old turns to make the request fit the context window. That stage was **removed** — the pipeline no longer runs any dropping/scoring logic, and the `RollingWindowConfig`, `IntelligentContextConfig`, and `ScoringWeights` classes are gone from the Python package (the TypeScript SDK's types still export the equivalent field names, but wired to nothing). Headroom now does _live-zone-only_ compression: it compresses content in place and leaves the message list intact. See [Context Management](https://docs.headroomlabs.ai/docs/context-management) . ### [ContentRouter](https://docs.headroomlabs.ai/docs/architecture#contentrouter) ContentRouter detects the type of each content block and dispatches it to exactly one compressor: | Detected content | Compressor | Typical savings | | --- | --- | --- | | JSON arrays (tool outputs) | SmartCrusher | 70–90% | | Source code | CodeAwareCompressor (opt-in; off by default) | 40–70% | | Search / grep results | SearchCompressor | 80–95% | | Build / test logs | LogCompressor | 85–95% | | Diffs | DiffCompressor | 40–80% | | HTML | HTMLExtractor (trafilatura) | 70–90% | | Tabular (CSV/TSV/markdown tables) | TabularCompressor | 60–90% | | Structured config (YAML/TOML/INI) | ConfigCompressor | 40–70% | | Plain text | TextCrusher | 30–60% | | Anything else | Kompress (ML fallback) | varies | To avoid recompressing the same content, ContentRouter keeps a **two-tier, TTL-bounded cache**: a _skip set_ of content already known not to compress, and a _result cache_ of previously compressed output. Default TTL is 30 minutes. ### [Rust core](https://docs.headroomlabs.ai/docs/architecture#rust-core) The heaviest compressors run in a native Rust extension — `headroom._core`, built with PyO3 — that the proxy loads at startup. SmartCrusher and the search/log/diff compressors and content detection are Rust-backed; the Python classes you import are thin, API-compatible shims over them. The ML fallback (**Kompress**, a ModernBERT token compressor) runs separately through ONNX Runtime, either locally or offloaded to a remote endpoint (see [Text & Logs](https://docs.headroomlabs.ai/docs/text-and-logs) ). [Compression modes and provider caches](https://docs.headroomlabs.ai/docs/architecture#compression-modes-and-provider-caches) ------------------------------------------------------------------------------------------------------------------------------ Headroom runs in one of two modes (`--mode`, default `cache`): * **`cache` mode (default)** — compresses only the newest delta in each turn and forwards prior turns byte-faithfully, so the provider's prefix cache is never invalidated mid-conversation. Best for coding agents and any long, multi-turn session. * **`token` mode** — prioritizes raw token removal and may recompress earlier turns, trading some cache stability for maximum savings. Because prefix caching is where most of the cost savings live on multi-turn workloads, keeping the stable prefix intact matters. What each provider's cache buys you: | Provider | Mechanism | Savings on cached tokens | | --- | --- | --- | | Anthropic | `cache_control` on the stable prefix | up to ~90% | | OpenAI | automatic prefix caching | up to ~50% | | Google | `CachedContent` API | up to ~75% | See [Cache Optimization](https://docs.headroomlabs.ai/docs/cache-optimization) and [Savings Profiles](https://docs.headroomlabs.ai/docs/proxy#savings-profiles) for how the two modes interact with the built-in profiles. [CCR: Compress-Cache-Retrieve](https://docs.headroomlabs.ai/docs/architecture#ccr-compress-cache-retrieve) ----------------------------------------------------------------------------------------------------------- Compression is **reversible**. When ContentRouter compresses a tool output, the original is stored in a local Compress-Cache-Retrieve (CCR) store. If the model needs the full data, it calls a `headroom_retrieve` tool and gets the original back. Compress: 1000 items -> 15 items (original stored in CCR) Cache: hash-indexed local SQLite store Retrieve: model calls headroom_retrieve("abc123") -> original 1000 items CCR is **on by default**. Disable the markers and the injected retrieval tool with `--no-ccr`, or run a marker-free, format-native lossless mode with `--lossless`. See [Reversible Compression (CCR)](https://docs.headroomlabs.ai/docs/ccr) . [TOIN: Tool Output Intelligence Network](https://docs.headroomlabs.ai/docs/architecture#toin-tool-output-intelligence-network) ------------------------------------------------------------------------------------------------------------------------------- TOIN learns which fields matter for a given tool over repeated calls — which items get retrieved, which fields carry signal — and feeds that back into SmartCrusher's importance scoring so compression gets sharper for the tools you actually use. TOIN is **local and observation-only**: it aggregates statistics on your own machine (or your own proxy instance). Nothing about your tools or traffic is shared across users or sent off the box. For a brand-new tool type it falls back to statistical heuristics and improves as it observes more calls. [What Headroom does not rewrite](https://docs.headroomlabs.ai/docs/architecture#what-headroom-does-not-rewrite) ---------------------------------------------------------------------------------------------------------------- * **Your prompt text** — the natural-language instructions you write are preserved. Compression targets bulk content blocks (tool outputs, file reads, logs), not your intent. * **System prompts** — preserved by default so the hottest part of the prefix cache stays stable. A savings profile can opt into compacting them. * **Code** — passes through unchanged unless AST-based code compression is explicitly enabled (off by default). * **Model responses** — returned from the provider unchanged. * **Short content** — blocks below the minimum-token threshold pass through (overhead would exceed savings). [API Reference\ \ Complete API reference for the Headroom Python and TypeScript SDKs. Core client, configuration types, result types, errors, and utilities.](https://docs.headroomlabs.ai/docs/api-reference) [CI/CD Flow Diagrams\ \ Visual decision trees for pull requests, release publishing, Docker images, docs deploys, and manual validation.](https://docs.headroomlabs.ai/docs/ci-cd-flows) ### On this page [High-level flow](https://docs.headroomlabs.ai/docs/architecture#high-level-flow) [Entry points](https://docs.headroomlabs.ai/docs/architecture#entry-points) [The compression pipeline](https://docs.headroomlabs.ai/docs/architecture#the-compression-pipeline) [ContentRouter](https://docs.headroomlabs.ai/docs/architecture#contentrouter) [Rust core](https://docs.headroomlabs.ai/docs/architecture#rust-core) [Compression modes and provider caches](https://docs.headroomlabs.ai/docs/architecture#compression-modes-and-provider-caches) [CCR: Compress-Cache-Retrieve](https://docs.headroomlabs.ai/docs/architecture#ccr-compress-cache-retrieve) [TOIN: Tool Output Intelligence Network](https://docs.headroomlabs.ai/docs/architecture#toin-tool-output-intelligence-network) [What Headroom does not rewrite](https://docs.headroomlabs.ai/docs/architecture#what-headroom-does-not-rewrite) --- # Benchmarks | Headroom Benchmarks ========== Compression performance, accuracy preservation, and latency overhead, measured on reproducible local benchmarks. Copy MarkdownOpen Headroom's core promise: compress context without losing accuracy. This page covers compression benchmarks, accuracy evaluations, and latency overhead. Every number below is measured locally and reproducible (see [Reproducing Results](https://docs.headroomlabs.ai/docs/benchmarks#reproducing-results) ). For local inference, the main benefit is often faster prompt processing rather than lower API spend. See [Local LLM prefill benchmarking](https://docs.headroomlabs.ai/docs/local-llm-prefill) for a reproducible passthrough-vs-optimized proxy workflow. [Compression Performance](https://docs.headroomlabs.ai/docs/benchmarks#compression-performance) ------------------------------------------------------------------------------------------------ Measured on Apple M-series (CPU), Headroom 0.37.0, via `python benchmarks/bench_latency.py` (10 warm iterations per scenario; latency is p50 compression overhead, not an LLM call). | Content Type | Original | Compressed | Saved | Ratio | Latency (p50) | | --- | --- | --- | --- | --- | --- | | JSON array — search results (100 items) | 10,200 | 5,300 | 4,900 | **48%** | 0.20ms | | JSON array — search results (500 items) | 50,200 | 25,800 | 24,400 | **49%** | 0.77ms | | Structured logs (100 entries) | 7,600 | 3,600 | 4,000 | **53%** | 0.17ms | | Structured logs (500 entries) | 35,700 | 16,200 | 19,400 | **54%** | 0.51ms | | Documentation text (20K tokens) | 20,100 | 1,600 | 18,400 | **92%** | 1.4ms | | Python source (~200 lines) | 2,600 | 2,600 | 0 | 0.0% | 1.3ms | | **Total** | **126,400** | **55,100** | **71,300** | **56%** | **4.4ms** | Zero compression is intentional Python source shows 0% compression: SmartCrusher only compresses JSON arrays, and code passes through to preserve correctness (verified: `headroom/transforms/smart_crusher.py:242`). [Accuracy Benchmarks](https://docs.headroomlabs.ai/docs/benchmarks#accuracy-benchmarks) ---------------------------------------------------------------------------------------- ### [HTML Extraction](https://docs.headroomlabs.ai/docs/benchmarks#html-extraction) **Dataset**: Scrapinghub Article Extraction Benchmark (181 HTML pages with ground truth), via `allenai/scrapinghub-article-extraction-benchmark` on Hugging Face. | Metric | Value | | --- | --- | | **F1 Score** | 0.919 | | **Precision** | 0.875 | | **Recall** | 0.985 | | **Compression** | 94.8% | For LLM applications, recall is critical -- 98.5% means nearly all article content is preserved. The slight precision drop (some extra content) does not hurt LLM accuracy. ### [JSON Compression (SmartCrusher)](https://docs.headroomlabs.ai/docs/benchmarks#json-compression-smartcrusher) **Test**: 100-entry JSON log array with one anomalous error entry (error code, resolution, and affected count) injected at position 67, compressed via the public `compress()` API with an OpenAI-format tool-result message. | Metric | Value | | --- | --- | | Input tokens (before) | 4,937 | | Input tokens (after) | 3,053 | | Compression | **38.2%** | | Error entry preserved in output | Yes (verified by substring check) | SmartCrusher preserves first N items (schema), last N items (recency), all anomalies (errors, warnings), and statistical distribution. This test confirms the anomalous entry survives compression; it does not include an LLM-graded "correct answers" comparison, which would require a paid API call to reproduce. ### [QA Accuracy Preservation](https://docs.headroomlabs.ai/docs/benchmarks#qa-accuracy-preservation) Requires a paid LLM call to reproduce This comparison (`tests/test_evals/test_html_oss_benchmarks.py::TestQAAccuracyPreservation`) runs SQuAD questions against both the original HTML and the extracted text using an LLM judge, and only executes when `OPENAI_API_KEY` is set. No committed result artifact for it exists in this repo, so no number is published here. Set `OPENAI_API_KEY` and run `pytest tests/test_evals/test_html_oss_benchmarks.py -k qa_accuracy -v -s` to reproduce it yourself. [Latency Overhead](https://docs.headroomlabs.ai/docs/benchmarks#latency-overhead) ---------------------------------------------------------------------------------- ### [SDK Compression Latency](https://docs.headroomlabs.ai/docs/benchmarks#sdk-compression-latency) Measured per-scenario on Apple M-series (CPU) via `python benchmarks/bench_latency.py --scenario json --iterations 10`. This is the local compression pipeline's own overhead — no LLM call is involved. | Scenario | Tokens In | Tokens Out | Saved | p50 (ms) | p95 (ms) | | --- | --- | --- | --- | --- | --- | | JSON: Search Results (100 items) | 10.2K | 5.3K | 4.9K | 0.20 | 0.21 | | JSON: Search Results (500 items) | 50.2K | 25.8K | 24.4K | 0.77 | 1.1 | | JSON: Search Results (1K items) | 100.5K | 51.7K | 48.8K | 1.4 | 1.6 | | JSON: API Responses (500 items) | 38.9K | 21.8K | 17.0K | 0.77 | 0.80 | | JSON: Database Rows (1K rows) | 43.7K | 16.2K | 27.5K | 0.78 | 0.81 | | JSON: String Array (100 strings) | 1.1K | 226 | 825 | 0.15 | 0.16 | | JSON: String Array (500 strings) | 4.9K | 228 | 4.6K | 0.11 | 0.12 | | JSON: Number Array (200 numbers) | 1.2K | 146 | 1.1K | 0.17 | 0.18 | | JSON: Mixed Array (250 items) | 2.3K | 1.1K | 1.2K | 0.24 | 0.25 | ### [Cost-Benefit Analysis](https://docs.headroomlabs.ai/docs/benchmarks#cost-benefit-analysis) Net latency benefit = LLM time saved from fewer tokens minus compression overhead (at Claude Sonnet 4.5's 0.03ms/token prefill rate, $3.0/MTok input pricing): | Scenario | Compress (ms) | LLM Saved (ms) | Net Benefit | Savings per 1K Requests | | --- | --- | --- | --- | --- | | JSON: Search Results (100 items) | 0.20 | 146 | **+145.8ms** | $14.60 | | JSON: Search Results (500 items) | 0.77 | 731 | **+730.2ms** | $73.09 | | JSON: Search Results (1K items) | 1.4 | 1,464 | **+1,462.6ms** | $146.40 | | JSON: API Responses (500 items) | 0.77 | 511 | **+510.2ms** | $51.10 | | JSON: Database Rows (1K rows) | 0.78 | 824 | **+822.9ms** | $82.36 | Compression paid for itself in latency for all 25 compressing scenarios measured (JSON, structured logs, agentic multi-turn, and long-document text) against Claude Sonnet 4.5. Slower and more expensive models (Opus) benefit even more, since the same compression overhead offsets a larger per-token prefill cost. ### [Pipeline Step Timing](https://docs.headroomlabs.ai/docs/benchmarks#pipeline-step-timing) Measured via the same `bench_latency.py` run's per-transform breakdown (content detection + routing is the dominant step in every JSON scenario): | Scenario | `content_router` p50 | % of total pipeline time | | --- | --- | --- | | JSON: String Array (500 strings) | 0.08ms | 68% | | JSON: Search Results (100 items) | 0.16ms | 82% | | JSON: Mixed Array (250 items) | 0.21ms | 85% | | JSON: Database Rows (1K rows) | 0.72ms | 95% | | JSON: Search Results (5K items) | 6.6ms | 99% | ContentRouter accounted for 68--99% of pipeline cost across the 14 JSON scenarios measured (mean ~87%), the rest going to tokenization and the compression transform itself. [Reproducing Results](https://docs.headroomlabs.ai/docs/benchmarks#reproducing-results) ---------------------------------------------------------------------------------------- git clone https://github.com/headroomlabs-ai/headroom.git cd headroom pip install -e ".[evals,html]" pytest tests/test_evals/ -v -s # Latency and compression-ratio tables above python benchmarks/bench_latency.py [Runtime Rollouts\ \ Deterministic runtime feature control for installed Headroom artifacts.](https://docs.headroomlabs.ai/docs/runtime-rollouts) [Limitations\ \ When Headroom helps, when it does not, and what to watch out for. Honest documentation of compression constraints and safety gates.](https://docs.headroomlabs.ai/docs/limitations) ### On this page [Compression Performance](https://docs.headroomlabs.ai/docs/benchmarks#compression-performance) [Accuracy Benchmarks](https://docs.headroomlabs.ai/docs/benchmarks#accuracy-benchmarks) [HTML Extraction](https://docs.headroomlabs.ai/docs/benchmarks#html-extraction) [JSON Compression (SmartCrusher)](https://docs.headroomlabs.ai/docs/benchmarks#json-compression-smartcrusher) [QA Accuracy Preservation](https://docs.headroomlabs.ai/docs/benchmarks#qa-accuracy-preservation) [Latency Overhead](https://docs.headroomlabs.ai/docs/benchmarks#latency-overhead) [SDK Compression Latency](https://docs.headroomlabs.ai/docs/benchmarks#sdk-compression-latency) [Cost-Benefit Analysis](https://docs.headroomlabs.ai/docs/benchmarks#cost-benefit-analysis) [Pipeline Step Timing](https://docs.headroomlabs.ai/docs/benchmarks#pipeline-step-timing) [Reproducing Results](https://docs.headroomlabs.ai/docs/benchmarks#reproducing-results) --- # Reversible Compression (CCR) | Headroom Reversible Compression (CCR) ============================ Compress-Cache-Retrieve architecture that makes compression lossless — the LLM can always get the original data back. Copy MarkdownOpen Headroom's CCR (Compress-Cache-Retrieve) architecture makes compression **reversible**. When content is compressed, the original data is cached locally. If the LLM needs the full data, it retrieves it instantly. Nothing is ever thrown away Unlike traditional lossy compression, CCR guarantees that every piece of original data remains accessible in the compression store, even when the compressed content the LLM sees is much smaller. Actual token savings depend heavily on how redundant the source data is -- see [How Compression Works](https://docs.headroomlabs.ai/docs/how-compression-works#content-type-detection) for measured ranges per content type. [The problem with traditional compression](https://docs.headroomlabs.ai/docs/ccr#the-problem-with-traditional-compression) --------------------------------------------------------------------------------------------------------------------------- Traditional compression forces a difficult tradeoff: * **Aggressive compression** risks losing data the LLM needs * **Conservative compression** misses out on token savings CCR eliminates this tradeoff entirely. Compress aggressively, retrieve on demand. [Architecture](https://docs.headroomlabs.ai/docs/ccr#architecture) ------------------------------------------------------------------- CCR flows through four phases: TOOL OUTPUT (1000 items) -> SmartCrusher compresses to 20 items -> Original cached with hash=abc123 -> Retrieval tool injected into context LLM PROCESSING Option A: LLM solves task with 20 items -> Done (90% savings) Option B: LLM calls headroom_retrieve(hash=abc123) -> Response Handler returns full data automatically [Phase 1: Compression Store](https://docs.headroomlabs.ai/docs/ccr#phase-1-compression-store) ---------------------------------------------------------------------------------------------- When SmartCrusher compresses tool output: 1. The original content is stored, keyed by hash, with LRU-style eviction once the store hits capacity. In the proxy, this store defaults to a SQLite file (`ccr_store.db` in the workspace directory) rather than a pure in-process dict -- so it survives a proxy restart and is shared across worker processes. Set `HEADROOM_CCR_BACKEND=memory` to opt back into an in-process-only store. Other backend names are resolved via the `headroom.ccr_backend` setuptools entry-point group (e.g. a `redis` backend, if one is installed and registered) -- none besides `sqlite`/`memory` ship in this repo. See `headroom/cache/compression_store.py:1000-1017`. 2. A hash key is generated for retrieval 3. A marker is added to the compressed output: [1000 items compressed to 20. Retrieve more: hash=abc123] [Phase 2: Tool Injection](https://docs.headroomlabs.ai/docs/ccr#phase-2-tool-injection) ---------------------------------------------------------------------------------------- Headroom injects a `headroom_retrieve` tool into the LLM's available tools: { "name": "headroom_retrieve", "description": "Retrieve original uncompressed data from Headroom cache", "parameters": { "hash": "The hash key from the compression marker" } } The LLM sees this tool alongside your application's tools and can call it whenever the compressed data is insufficient. [Phase 3: Response Handler](https://docs.headroomlabs.ai/docs/ccr#phase-3-response-handler) -------------------------------------------------------------------------------------------- When the LLM calls `headroom_retrieve`: 1. The Response Handler intercepts the tool call 2. Data is retrieved from the local cache (around 1ms) 3. The result is added to the conversation 4. The API call continues automatically The client never sees CCR tool calls on the Anthropic and OpenAI proxy paths; Headroom resolves them transparently there. Gemini CCR boundary Buffered native Gemini requests resolve `headroom_retrieve` server-side and return the model's final response. Streaming native Gemini requests keep the existing forwarding behavior. When a response contains `headroom_retrieve` alongside a client-owned function call, Headroom preserves both calls for the client instead of resolving the mixed response. Google's OpenAI-compatible Gemini endpoint can also return `finish_reason=MALFORMED_FUNCTION_CALL` on large function-response continuations after CCR retrieval; that separate limitation remains tracked in [issue #2041](https://github.com/headroomlabs-ai/headroom/issues/2041) . [Phase 4: Context Tracker](https://docs.headroomlabs.ai/docs/ccr#phase-4-context-tracker) ------------------------------------------------------------------------------------------ Across multiple turns, the Context Tracker maintains awareness of all compressed content: 1. Remembers what was compressed in earlier turns 2. Analyzes new queries for relevance to compressed content 3. Proactively expands relevant data before the LLM asks Turn 1: User searches for files -> 500 files compressed to 15, cached (hash=abc123) -> LLM answers with 15 files Turn 5: User asks "What about the auth middleware?" -> Context Tracker detects "auth" may match cached content -> Proactively expands compressed data -> LLM finds auth_middleware.py in the full list [Retrieving originals](https://docs.headroomlabs.ai/docs/ccr#retrieving-originals) ----------------------------------------------------------------------------------- CCR works automatically on the Anthropic and OpenAI proxy paths, but you can also retrieve cached data programmatically: TypeScriptPython import { compress } from "headroom-ai"; import type { CCRConfig } from "headroom-ai"; // CCR is enabled by default when compressing through the proxy. const result = await compress(messages, { model: "gpt-4o", }); // Access compressed messages — CCR markers are embedded automatically console.log(result.messages); // CCR configuration options const ccrConfig: CCRConfig = { enabled: true, injectTool: true, // Inject headroom_retrieve tool injectRetrievalMarker: true, // Add retrieval markers to compressed output feedbackEnabled: true, // Learn from retrieval patterns storeMaxEntries: 1000, // Max cached items storeTtlSeconds: 3600, // Cache TTL }; The full CCR loop -- markers injected, `headroom_retrieve` tool added to the request, and the model's retrieval call resolved automatically -- only runs through `headroom proxy` (`headroom/proxy/handlers/*.py` wire tool injection and the response handler; the direct SDK client does not). Point an OpenAI client at the proxy to get it: from openai import OpenAI # Start `headroom proxy` first (defaults to port 8787); use --no-optimize # for a passthrough baseline or --no-ccr to disable CCR specifically. client = OpenAI(base_url="http://127.0.0.1:8787/v1", api_key="local") response = client.chat.completions.create( model="gpt-4o", messages=messages, ) `HeadroomClient` (the direct SDK integration, no proxy) still routes JSON tool outputs through SmartCrusher and embeds `<>` markers when a compression ratio warrants one, and the original is stored and retrievable via `headroom.cache.compression_store`. It does **not**, however, inject the `headroom_retrieve` tool or intercept the model's retrieval call for you -- that wiring lives only in the proxy's request handlers. Resolve markers yourself with `headroom.ccr.marker_resolution.resolve_markers_in_text` if you're integrating CCR outside the proxy. [Retention](https://docs.headroomlabs.ai/docs/ccr#retention) ------------------------------------------------------------- Proxy CCR originals are kept for 1800 seconds (30 minutes) by default. For longer autonomous agent runs, set `HEADROOM_CCR_TTL_SECONDS` before starting the proxy: HEADROOM_CCR_TTL_SECONDS=7200 headroom proxy Check the effective setting at `/v1/retrieve/stats` under `store.default_ttl_seconds`. [Message-level CCR](https://docs.headroomlabs.ai/docs/ccr#message-level-ccr) ----------------------------------------------------------------------------- > **Retired:** The "Message-level CCR via IntelligentContext" feature (where `IntelligentContext` would store dropped messages in CCR with a retrieval marker) was part of the `IntelligentContextConfig` API that has since been removed. Context management is now handled automatically by the pipeline without a separate configurable IntelligentContext stage. Tool-output CCR via SmartCrusher and ContentRouter remains fully supported. [CCR-enabled components](https://docs.headroomlabs.ai/docs/ccr#ccr-enabled-components) --------------------------------------------------------------------------------------- | Component | What it compresses | CCR integration | | --- | --- | --- | | **SmartCrusher** | JSON arrays (tool outputs) | Stores original array, marker includes hash | | **ContentRouter** | Code, logs, search results, text | Stores original content by strategy | [Why CCR matters](https://docs.headroomlabs.ai/docs/ccr#why-ccr-matters) ------------------------------------------------------------------------- | Approach | Risk | Savings | | --- | --- | --- | | No compression | None | 0% | | Traditional compression | Data loss | Depends on content and target ratio | | CCR compression | None (reversible) | Same range as traditional compression -- CCR changes the risk, not the ratio | CCR gives you the savings of aggressive compression with zero risk: SmartCrusher, LogCompressor, SearchCompressor, and Kompress can all compress as aggressively as their configuration allows, because the original is always one retrieval away. See [How Compression Works](https://docs.headroomlabs.ai/docs/how-compression-works#content-type-detection) for measured savings ranges per compressor. [Text & Log Compression\ \ Specialized compressors for search results, build logs, diffs, and general text. Each preserves what matters for its content type.](https://docs.headroomlabs.ai/docs/text-and-logs) [Cache Optimization\ \ Stabilize message prefixes for provider KV cache hits and configure provider-specific caching strategies.](https://docs.headroomlabs.ai/docs/cache-optimization) ### On this page [The problem with traditional compression](https://docs.headroomlabs.ai/docs/ccr#the-problem-with-traditional-compression) [Architecture](https://docs.headroomlabs.ai/docs/ccr#architecture) [Phase 1: Compression Store](https://docs.headroomlabs.ai/docs/ccr#phase-1-compression-store) [Phase 2: Tool Injection](https://docs.headroomlabs.ai/docs/ccr#phase-2-tool-injection) [Phase 3: Response Handler](https://docs.headroomlabs.ai/docs/ccr#phase-3-response-handler) [Phase 4: Context Tracker](https://docs.headroomlabs.ai/docs/ccr#phase-4-context-tracker) [Retrieving originals](https://docs.headroomlabs.ai/docs/ccr#retrieving-originals) [Retention](https://docs.headroomlabs.ai/docs/ccr#retention) [Message-level CCR](https://docs.headroomlabs.ai/docs/ccr#message-level-ccr) [CCR-enabled components](https://docs.headroomlabs.ai/docs/ccr#ccr-enabled-components) [Why CCR matters](https://docs.headroomlabs.ai/docs/ccr#why-ccr-matters) --- # Cache Optimization | Headroom Cache Optimization ================== Stabilize message prefixes for provider KV cache hits and configure provider-specific caching strategies. Copy MarkdownOpen LLM providers cache prompt prefixes to avoid reprocessing identical input on repeated calls. Headroom's **CacheAligner** is detector-only, so it surfaces prefix drift, reports observability data, and leaves message assembly to the caller. CacheAligner is opt-in and detector-only CacheAligner is **disabled by default** and hard-disabled inside the proxy — it never runs on your traffic unless you explicitly enable it, and even then it only _reports_ metrics; it never repairs a prefix. In proxy mode, prefix-cache stability comes from **cache mode** (`--mode cache`, the default), which compresses only the newest delta and forwards prior turns byte-faithfully. See [Savings Profiles](https://docs.headroomlabs.ai/docs/proxy#savings-profiles) . [What CacheAligner reports](https://docs.headroomlabs.ai/docs/cache-optimization#what-cachealigner-reports) ------------------------------------------------------------------------------------------------------------ System prompts often contain dynamic content, such as dates, session IDs, and timestamps, that changes between requests. Even a single character difference at the start of a prompt invalidates the entire provider cache. CacheAligner does not extract, move, normalize, reorder, strip, compress, or rewrite content. It detects volatile content and reports the stable prefix hash plus cache metrics so you can fix the prefix at the source: | Signal | Meaning | | --- | --- | | `warnings` | The prefix contains unstable content | | `cache_metrics.stable_prefix_bytes` | Stable prefix size in bytes | | `cache_metrics.stable_prefix_tokens_est` | Stable prefix size in estimated tokens | | `cache_metrics.stable_prefix_hash` | Stable prefix hash for repeated-wake comparison | | `cache_metrics.prefix_changed` | The prefix drifted since the previous wake | | `markers` | The emitted `stable_prefix_hash` marker for observability | The prefix must stay byte-identical across requests for provider KV caches to reuse previously computed attention states. [Provider-specific strategies](https://docs.headroomlabs.ai/docs/cache-optimization#provider-specific-strategies) ------------------------------------------------------------------------------------------------------------------ Each LLM provider implements caching differently. Headroom applies the optimal strategy for each. ### [Anthropic](https://docs.headroomlabs.ai/docs/cache-optimization#anthropic) Anthropic supports explicit `cache_control` blocks that mark content as cacheable. Cached input tokens cost **90% less** than regular input tokens. Keep the stable prefix byte-identical, then place provider cache markers where your client or orchestrator already assembles the request. Headroom's job is to surface prefix instability, not repair it. | Metric | Value | | --- | --- | | Cache read discount | 90% off input price | | Cache write cost | 25% premium on first write | | Cache TTL | 5 minutes (extended on hit) | ### [OpenAI](https://docs.headroomlabs.ai/docs/cache-optimization#openai) OpenAI uses automatic **prefix caching**. If consecutive requests share the same message prefix, the provider reuses cached KV states. No explicit API markers are needed, but the prefix must be byte-identical. CacheAligner tells you when the prefix changed, which is the only signal you need to keep OpenAI prefix caching effective. | Metric | Value | | --- | --- | | Cache read discount | 50% off input price | | Activation | Automatic (prefix match) | | Min prefix length | 1024 tokens | ### [Google](https://docs.headroomlabs.ai/docs/cache-optimization#google) Google provides the **CachedContent API**, which lets you explicitly cache large context (system instructions, documents, tools) and reference it across requests. Cached tokens cost **75% less**. Keep the prefix stable in your integration layer; Headroom reports when the cacheable zone drifts. The CachedContent lifecycle itself also stays in your integration layer. | Metric | Value | | --- | --- | | Cache read discount | 75% off input price | | Mechanism | Explicit CachedContent API objects | | Min cache size | 32,768 tokens | [What this means in practice](https://docs.headroomlabs.ai/docs/cache-optimization#what-this-means-in-practice) ---------------------------------------------------------------------------------------------------------------- Keep the stable prefix first, keep volatile content out of it, and treat CacheAligner warnings as a signal that the caller needs to move assembly logic. CacheAligner surfaces prefix instability, provider caches reward byte-identical prefixes, and the caller owns the actual message layout. [Cold-prefix recompaction (when the cache lapses)](https://docs.headroomlabs.ai/docs/cache-optimization#cold-prefix-recompaction-when-the-cache-lapses) -------------------------------------------------------------------------------------------------------------------------------------------------------- Byte-identical forwarding only pays off while the prompt cache is warm. When a session goes idle past the provider's cache TTL, that cache is dead — so re-sending the prefix verbatim buys nothing, and it's the one safe moment to rewrite it. The **cold-prefix hook** detects a lapsed cache and recompacts the whole prefix (cross-turn dedupe + superseded-read drop + lossless folds) instead, then re-caches the smaller result. It only fires on a _confirmed_\-cold turn — a wrong call would bust a warm cache — so it reads the real TTL (exact for Claude Code via its cache-control env vars; learned over time for other providers). For models that re-send **reasoning** as plain text every turn (Kimi/GLM/DeepSeek), a companion hook Kompresses that reasoning on warm turns and drops it on cold ones. Both are off by default. See [Cold-prefix hook & reasoning compaction](https://docs.headroomlabs.ai/docs/configuration#cold-prefix-hook--reasoning-compaction) for the exact flags and when each is safe to enable. [Reversible Compression (CCR)\ \ Compress-Cache-Retrieve architecture that makes compression lossless — the LLM can always get the original data back.](https://docs.headroomlabs.ai/docs/ccr) [Agent Orchestration\ \ Keep repeated agent wakes cache-friendly while using CCR for lossless memory digest retrieval.](https://docs.headroomlabs.ai/docs/agent-orchestration) ### On this page [What CacheAligner reports](https://docs.headroomlabs.ai/docs/cache-optimization#what-cachealigner-reports) [Provider-specific strategies](https://docs.headroomlabs.ai/docs/cache-optimization#provider-specific-strategies) [Anthropic](https://docs.headroomlabs.ai/docs/cache-optimization#anthropic) [OpenAI](https://docs.headroomlabs.ai/docs/cache-optimization#openai) [Google](https://docs.headroomlabs.ai/docs/cache-optimization#google) [What this means in practice](https://docs.headroomlabs.ai/docs/cache-optimization#what-this-means-in-practice) [Cold-prefix recompaction (when the cache lapses)](https://docs.headroomlabs.ai/docs/cache-optimization#cold-prefix-recompaction-when-the-cache-lapses) --- # CI/CD Flow Diagrams | Headroom CI/CD Flow Diagrams =================== Visual decision trees for pull requests, release publishing, Docker images, docs deploys, and manual validation. Copy MarkdownOpen [Purpose](https://docs.headroomlabs.ai/docs/ci-cd-flows#purpose) ----------------------------------------------------------------- This page is the quick visual map for Headroom automation. Use it when opening a PR, reviewing a PR, cutting a release, or deciding which workflow owns a failure. The short version: * Pull requests are gated by PR governance, path-filtered CI, targeted e2e workflows, and human review. * Release publishing is not triggered by every merge to `main`. `release-please` maintains a release PR; merging that release PR creates the tag and GitHub Release that trigger publishing. * Docker images are built as multi-architecture digests first, then merged into tagged manifests. * Docs are deployed by Vercel's own Git integration, not by a GitHub Actions job -- `.github/workflows/docs.yml` ("Validate Docs") only builds the Next.js/Fumadocs app on pull requests touching `docs/**` to catch a broken build before merge; it deploys nothing. [PR Flow](https://docs.headroomlabs.ai/docs/ci-cd-flows#pr-flow) ----------------------------------------------------------------- flowchart TD A[Open, edit, synchronize, or mark PR ready] --> B[PR Governance] B --> B1{Template complete and ready?} B1 -- no --> B2[Add needs author action label and governance comment] B1 -- yes --> B3[Add ready for review label when no blocking status exists] A --> C[Path filters decide workflow surface] C --> D{Code paths changed?} D -- yes --> E[CI changes job] E --> F[lint: ruff, format, mypy] E --> G[build-wheel: Rust extension wheel] E --> H[prefetch-model: Hugging Face cache] G --> I[test shards 1 to 4] H --> I G --> J[test-extras, test-agno, dashboard UI] E --> K[build: release-profile wheel and sdist smoke] C --> L{E2E paths changed?} L -- yes --> M[Docker native e2e and platform wrapper checks] C --> N{Init or wrap paths changed?} N -- yes --> O[Init E2E, Wrap E2E, native init or wrap smoke tests] C --> P{Rust paths changed?} P -- yes --> Q[Rust fmt, clippy, tests, wheel build, audit] C --> R{Devcontainer paths changed?} R -- yes --> S[Devcontainer validation and linked worktree smoke] C --> T{Release-critical paths changed?} T -- yes --> U[Release dry-run: wheel matrix and smoke-import gates] C --> V{Workflow files changed?} V -- yes --> W[Workflow validation with actionlint and act dry-run] F --> X{All required checks green?} I --> X J --> X K --> X M --> X O --> X Q --> X S --> X U --> X W --> X B3 --> X X -- no --> Y[Fix, rebase, or request changes] X -- yes --> Z[Human review and merge when approved] [PR Decision Tree](https://docs.headroomlabs.ai/docs/ci-cd-flows#pr-decision-tree) ----------------------------------------------------------------------------------- flowchart TD A[Review an open PR] --> B{Draft?} B -- yes --> C[Do not approve. Comment with remaining readiness steps.] B -- no --> D{Governance label says needs author action?} D -- yes --> E[Fix or ask author to complete template and real behavior proof] D -- no --> F{Merge state dirty or behind?} F -- dirty --> G[Resolve conflicts before reviewing final code] F -- behind --> H[Rebase or update branch, then rerun checks] F -- clean or unknown --> I{Any failing check?} I -- yes --> J[Read failing logs, classify as stale-main, infra, or code bug] J --> K{Can maintainer safely fix without changing author intent?} K -- yes --> L[Patch, test locally, push with lease] K -- no --> M[Request changes with exact file and line references] I -- no --> N{Code review complete?} N -- no --> O[Review diff, tests, docs, and behavior proof] N -- yes --> P[Approve] [Fork Workflow Approval](https://docs.headroomlabs.ai/docs/ci-cd-flows#fork-workflow-approval) ----------------------------------------------------------------------------------------------- GitHub may leave product workflows in `action_required` for first-time or fork contributors. Approve only after the diff is safe enough to execute in CI. flowchart TD A[Fork PR has action_required workflows] --> B{Diff is understandable and not suspicious?} B -- no --> C[Do not approve. Ask for changes or close if unsafe.] B -- yes --> D{Workflow runs use pull_request with read-scoped token?} D -- no --> E[Inspect workflow permissions before approving] D -- yes --> F[Approve queued CI runs] F --> G[Wait for fresh check results on latest head SHA] [Release Flow](https://docs.headroomlabs.ai/docs/ci-cd-flows#release-flow) --------------------------------------------------------------------------- flowchart TD A[Merge ordinary PR to main] --> B[Release Please on push to main] B --> C{Commit is releasable?} C -- docs, ci, chore only --> D[No release PR change] C -- fix, feat, breaking change --> E[Create or update Release PR] E --> F[Release PR contains version bump and changelog] F --> G{Ready to ship?} G -- no --> H[Keep merging regular PRs; bot updates Release PR] G -- yes --> I[Merge Release PR] I --> J[release-please tags vX.Y.Z and publishes GitHub Release] J --> K[release.yml starts on release: published] K --> L[detect-version] L --> M[build: sync versions, verify versions, changelog, npm packs] M --> N[build-wheels matrix] N --> O[collect-dist] N --> P[smoke-import wheels] O --> Q{Smoke import green?} P --> Q Q -- no --> R[Stop before publishing broken wheels] Q -- yes --> S[publish PyPI] M --> T[publish npm packages] M --> U[publish GitHub Package Registry packages] P --> V[publish Docker images through docker.yml] S --> W{PyPI published or PYPI_SKIP=true?} W -- no --> X[Do not update release assets] W -- yes --> Y[Create or update GitHub Release assets and notes] T --> Y U --> Y V --> Y [Release Decision Tree](https://docs.headroomlabs.ai/docs/ci-cd-flows#release-decision-tree) --------------------------------------------------------------------------------------------- flowchart TD A[Need a release?] --> B{Release PR exists?} B -- no --> C[Merge at least one releasable conventional commit to main] B -- yes --> D{Release PR checks green and changelog correct?} D -- no --> E[Fix source PRs or release config, then let release-please update] D -- yes --> F{Registry skip variables needed?} F -- yes --> G[Set PYPI_SKIP, NPM_SKIP, or GH_PACKAGES_SKIP deliberately] F -- no --> H[Merge Release PR] G --> H H --> I[Watch release.yml] I --> J{Failure before publish?} J -- yes --> K[Fix and rerun before any package is public] J -- no --> L{Failure after partial publish?} L -- yes --> M[Use rerun or skip variables to reach consistent GitHub Release state] L -- no --> N[Release complete] [Docker Publish Flow](https://docs.headroomlabs.ai/docs/ci-cd-flows#docker-publish-flow) ----------------------------------------------------------------------------------------- `docker.yml` can run directly on `push`, `workflow_dispatch`, or `release: published`, and it is also called by `release.yml`. flowchart TD A[Docker workflow starts] --> B[Matrix: variant x architecture] B --> C[Build each platform image by digest only] C --> D[Smoke-test image imports pydantic_core and headroom._core] D --> E{Smoke test green?} E -- no --> F[Stop before tag manifest] E -- yes --> G[Upload digest marker] G --> H[Per-variant manifest merge] H --> I[Apply tags to multi-arch manifest] I --> J{Release event?} J -- yes --> K[Version tags and latest retag rules] J -- no --> L[Branch, PR, dev, or manual tags as configured] [Docs Validate + Deploy Flow](https://docs.headroomlabs.ai/docs/ci-cd-flows#docs-validate--deploy-flow) -------------------------------------------------------------------------------------------------------- `.github/workflows/docs.yml` ("Validate Docs") only validates -- it has no deploy step. Deployment of the Next.js/Fumadocs site at `docs.headroomlabs.ai` is owned entirely by Vercel's own Git integration, outside GitHub Actions (`.github/workflows/docs.yml:1-9`). flowchart TD A[Docs change in PR] --> B{docs/** or docs.yml changed?} B -- no --> C[Validate Docs workflow does not run] B -- yes --> D[Validate Docs workflow: npm ci and npm run build] D --> E{Build green?} E -- no --> F[Fix before merge] E -- yes --> G[PR review and normal checks] G --> H[Merge to main] H --> I[Vercel Git integration builds and deploys, independent of GitHub Actions] [Manual Validation Flow](https://docs.headroomlabs.ai/docs/ci-cd-flows#manual-validation-flow) ----------------------------------------------------------------------------------------------- Use this when editing workflows or release automation. flowchart TD A[Edit workflow or release scripts] --> B[Run local workflow validation] B --> C[actionlint] B --> D[act dry-run fixtures] C --> E{Local validation green?} D --> E E -- no --> F[Fix before opening or updating PR] E -- yes --> G[Push PR] G --> H[workflow-validation job reruns same validation in CI] Recommended local command: bash scripts/validate-workflows.sh For release dry-runs: act workflow_dispatch -W .github/workflows/release.yml -e .github/act/dry-run.json [Gate Summary](https://docs.headroomlabs.ai/docs/ci-cd-flows#gate-summary) --------------------------------------------------------------------------- | Flow | Trigger | Main gates | Success condition | | --- | --- | --- | --- | | PR governance | `pull_request_target`, schedule, manual | Template, readiness labels, merge state, check labels | PR has no governance blockers | | CI | PR, push to `main`, manual | Path filter, lint, mypy, wheel build, model prefetch, test shards, package smoke | Required jobs green or path-skipped | | Rust | Rust paths, schedule | fmt, clippy, cargo test, wheel build, audit | Rust checks green; nightly parity is allowed to fail in Phase 0 | | E2E | CLI, install, wrap, Docker, package paths | Docker init/wrap, native init/wrap/install, platform smoke | Relevant lifecycle checks green | | Release dry-run | PRs touching release-critical paths | Version detection, wheel matrix, smoke imports | Publish path can build before merge | | Release publish | GitHub Release published by release-please | Version sync, changelog, wheels, smoke import, PyPI gate, npm, GPR, Docker | Public packages and GitHub Release assets are consistent | | Docs validate | PR touching `docs/**`, manual | Next.js/Fumadocs build | Build succeeds; deploy itself is done by Vercel's Git integration, not this workflow | [Workflow Ownership](https://docs.headroomlabs.ai/docs/ci-cd-flows#workflow-ownership) --------------------------------------------------------------------------------------- | Workflow | Owns | | --- | --- | | `.github/workflows/pr-health.yml` | PR body governance, readiness labels, rebase/conflict/failing-check labels | | `.github/workflows/ci.yml` | Python lint, type checks, wheel build, test shards, package smoke, workflow validation | | `.github/workflows/rust.yml` | Rust workspace quality gates and native wheel smoke artifacts | | `.github/workflows/init-e2e.yml` | Dockerized `headroom init` behavior | | `.github/workflows/wrap-e2e.yml` | Dockerized `headroom wrap` behavior | | `.github/workflows/init-native-e2e.yml` | Host-specific `headroom init -g` smoke tests | | `.github/workflows/install-native-e2e.yml` | Host-specific install CLI smoke tests | | `.github/workflows/wrap-native-e2e.yml` | Host-specific wrap prepare-only smoke tests | | `.github/workflows/devcontainers.yml` | Devcontainer startup and linked worktree compatibility | | `.github/workflows/release-please.yml` | Release PR aggregation from conventional commits | | `.github/workflows/release.yml` | Release build, wheel smoke-import gates, registry publishing, GitHub Release assets | | `.github/workflows/docker.yml` | GHCR multi-architecture image builds and manifests | | `.github/workflows/docs.yml` | Validates the Next.js/Fumadocs build on docs PRs; deploys nothing -- Vercel's own Git integration owns the actual deploy | [Architecture\ \ How Headroom compresses LLM traffic — from request interception through the ContentRouter compression pipeline to provider cache optimization.](https://docs.headroomlabs.ai/docs/architecture) [Releases & CI/CD\ \ Automated release pipeline with release-please, semantic versioning, multi-package publishing, and changelog generation.](https://docs.headroomlabs.ai/docs/releases) ### On this page [Purpose](https://docs.headroomlabs.ai/docs/ci-cd-flows#purpose) [PR Flow](https://docs.headroomlabs.ai/docs/ci-cd-flows#pr-flow) [PR Decision Tree](https://docs.headroomlabs.ai/docs/ci-cd-flows#pr-decision-tree) [Fork Workflow Approval](https://docs.headroomlabs.ai/docs/ci-cd-flows#fork-workflow-approval) [Release Flow](https://docs.headroomlabs.ai/docs/ci-cd-flows#release-flow) [Release Decision Tree](https://docs.headroomlabs.ai/docs/ci-cd-flows#release-decision-tree) [Docker Publish Flow](https://docs.headroomlabs.ai/docs/ci-cd-flows#docker-publish-flow) [Docs Validate + Deploy Flow](https://docs.headroomlabs.ai/docs/ci-cd-flows#docs-validate--deploy-flow) [Manual Validation Flow](https://docs.headroomlabs.ai/docs/ci-cd-flows#manual-validation-flow) [Gate Summary](https://docs.headroomlabs.ai/docs/ci-cd-flows#gate-summary) [Workflow Ownership](https://docs.headroomlabs.ai/docs/ci-cd-flows#workflow-ownership) --- # Claude Code on Azure AI Foundry | Headroom Claude Code on Azure AI Foundry =============================== Run Claude Code against Claude models on Azure AI Foundry, with Headroom compressing your prompts — fewer input tokens, same answers, your own Azure credentials. Copy MarkdownOpen If your Claude models live on **Azure AI Foundry**, you can still get Headroom's prompt compression. Headroom sits between Claude Code and Azure: it shrinks the big stuff in each request (file reads, logs, tool output) and forwards the rest to Azure using **your own Azure credentials**. You keep your Azure setup; Headroom just makes each call cheaper. [What you get](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#what-you-get) ----------------------------------------------------------------------------------------- * **Fewer input tokens** on every Claude Code request to Azure AI Foundry — large file reads, logs, and tool output are compressed before forwarding — so you pay for less. * **Same answers** — compression is reversible and content-aware. * **No new secrets** — Headroom never holds your Azure credentials. Claude Code keeps authenticating to Azure AI Foundry with its own `api-key` or Entra Bearer token; Headroom passes it through. [Before you start](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#before-you-start) ------------------------------------------------------------------------------------------------- You should already have Claude Code working against Azure AI Foundry **without** Headroom. That means these are set in your shell (or your `~/.claude/settings.json` `env` block): export CLAUDE_CODE_USE_FOUNDRY=1 export ANTHROPIC_FOUNDRY_RESOURCE= # e.g. ANTHROPIC_FOUNDRY_RESOURCE=my-org-claude No `ANTHROPIC_API_KEY` is needed — Foundry mode uses your Azure credentials. [Run it (one command)](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#run-it-one-command) ------------------------------------------------------------------------------------------------------- pip install "headroom-ai[proxy]" headroom wrap claude That's it. Because `CLAUDE_CODE_USE_FOUNDRY=1` is set, `headroom wrap claude` automatically: 1. derives your Azure AI Foundry endpoint from `ANTHROPIC_FOUNDRY_RESOURCE`, 2. starts the Headroom proxy with that endpoint as the upstream, 3. points Claude Code's Foundry endpoint at the proxy (`ANTHROPIC_FOUNDRY_BASE_URL`), 4. leaves your Azure resource, model, and credentials untouched. You'll see a line like: Foundry mode: ANTHROPIC_FOUNDRY_BASE_URL=http://127.0.0.1:8787/anthropic → upstream https://my-org-claude.services.ai.azure.com/anthropic Use Claude Code exactly as you normally would. [How it works](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#how-it-works) ----------------------------------------------------------------------------------------- Claude Code ──(Foundry request)──▶ Headroom ──(compressed)──▶ Azure AI Foundry (Claude) in Foundry mode compresses your resource (your api-key / Entra) ──────── passed through ───────────▶ authenticates you Claude Code sends its Foundry request (Anthropic API format) to Headroom. Headroom compresses the messages, then forwards to your Azure AI Foundry resource endpoint — `https://{ANTHROPIC_FOUNDRY_RESOURCE}.services.ai.azure.com/anthropic` — with your auth headers passed through unchanged. [If you have ANTHROPIC\_FOUNDRY\_BASE\_URL set explicitly](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#if-you-have-anthropic_foundry_base_url-set-explicitly) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ If your environment already has `ANTHROPIC_FOUNDRY_BASE_URL` set to the full Azure endpoint URL, Headroom uses it directly and `ANTHROPIC_FOUNDRY_RESOURCE` is not needed. Either configuration works. [Check that compression is working](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#check-that-compression-is-working) ----------------------------------------------------------------------------------------------------------------------------------- 1. Open the dashboard: [http://localhost:8787/dashboard](http://localhost:8787/dashboard) . "Tokens saved" should climb as you use Claude Code. 2. Or check response headers: `x-headroom-tokens-before`, `x-headroom-tokens-after`, `x-headroom-tokens-saved`. [Troubleshooting](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#troubleshooting) ----------------------------------------------------------------------------------------------- **Headroom says "ANTHROPIC\_BASE\_URL" instead of "Foundry mode"** `CLAUDE_CODE_USE_FOUNDRY` is not set in the shell where you ran `headroom wrap claude`. Make sure to export it before running, or set it in your shell profile. **Claude Code fails with a 401 / auth error** Your Azure credentials are not being forwarded correctly. Verify that Claude Code works against Azure AI Foundry directly (without Headroom) before wrapping. If it works direct but not through Headroom, open an issue with the proxy log. **"tokens saved" is always 0** Check the [dashboard](http://localhost:8787/dashboard) — if requests are flowing but savings are 0, content may be below the compression threshold or the Rust extension may not be installed (`pip install "headroom-ai[proxy]"` includes it). [Claude Code on Vertex AI\ \ Run Claude Code against Claude models on Google Vertex AI, with Headroom compressing your prompts — fewer input tokens, same answers, your own GCP login.](https://docs.headroomlabs.ai/docs/claude-code-vertex) [Use Headroom with Claude Code in VS Code\ \ Route the official Claude Code extension through Headroom's local compression proxy.](https://docs.headroomlabs.ai/docs/vscode-claude-code) ### On this page [What you get](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#what-you-get) [Before you start](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#before-you-start) [Run it (one command)](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#run-it-one-command) [How it works](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#how-it-works) [If you have ANTHROPIC\_FOUNDRY\_BASE\_URL set explicitly](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#if-you-have-anthropic_foundry_base_url-set-explicitly) [Check that compression is working](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#check-that-compression-is-working) [Troubleshooting](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry#troubleshooting) --- # Claude Code on Vertex AI | Headroom Claude Code on Vertex AI ======================== Run Claude Code against Claude models on Google Vertex AI, with Headroom compressing your prompts — fewer input tokens, same answers, your own GCP login. Copy MarkdownOpen If your Claude models live on **Google Vertex AI**, you can still get Headroom's prompt compression. Headroom sits between Claude Code and Vertex: it shrinks the big stuff in each request (file reads, logs, tool output) and forwards the rest to Vertex using **your own Google credentials**. You keep your GCP setup; Headroom just makes each call cheaper. [What you get](https://docs.headroomlabs.ai/docs/claude-code-vertex#what-you-get) ---------------------------------------------------------------------------------- * **Fewer input tokens** on every Claude Code request to Vertex — large file reads, logs, and tool output are compressed before forwarding — so you pay Vertex for less. * **Same answers** — compression is reversible and content-aware. * **No new secrets** — Headroom never holds your Google credentials. Claude Code keeps authenticating to Vertex with its own ADC token; Headroom passes it through. [Before you start](https://docs.headroomlabs.ai/docs/claude-code-vertex#before-you-start) ------------------------------------------------------------------------------------------ You should already have Claude Code working against Vertex **without** Headroom. That means these are set in your shell: export CLAUDE_CODE_USE_VERTEX=1 export ANTHROPIC_VERTEX_PROJECT_ID= export CLOUD_ML_REGION=us-east5 # your Vertex region (or "global") gcloud auth application-default login # or set GOOGLE_APPLICATION_CREDENTIALS No `ANTHROPIC_API_KEY` is needed — Vertex mode uses your Google login. [Run it (one command)](https://docs.headroomlabs.ai/docs/claude-code-vertex#run-it-one-command) ------------------------------------------------------------------------------------------------ pip install "headroom-ai[proxy]" headroom wrap claude That's it. Because `CLAUDE_CODE_USE_VERTEX=1` is set, `headroom wrap claude` automatically: 1. starts the Headroom proxy, 2. points Claude Code's Vertex endpoint at it (`ANTHROPIC_VERTEX_BASE_URL`), 3. leaves your project, region, and Google login untouched. You'll see a line like: Vertex mode: ANTHROPIC_VERTEX_BASE_URL=http://127.0.0.1:8787 → compress, then forward to Vertex with your GCP ADC token Use Claude Code exactly as you normally would. [How it works](https://docs.headroomlabs.ai/docs/claude-code-vertex#how-it-works) ---------------------------------------------------------------------------------- Claude Code ──(Vertex request)──▶ Headroom ──(compressed)──▶ Vertex AI (Claude) in Vertex mode compresses your project + region (your ADC token) ───────────── passed through ───────────▶ authenticates you Claude Code sends its normal Vertex `…:rawPredict` / `:streamRawPredict` request to Headroom. Headroom compresses the messages (keeping the Vertex request shape intact), then forwards to the correct regional Vertex host — derived from the request itself, so multi-region and `global` both work — using the Google token Claude Code already attached. [Check that compression is working](https://docs.headroomlabs.ai/docs/claude-code-vertex#check-that-compression-is-working) ---------------------------------------------------------------------------------------------------------------------------- 1. Open the dashboard: [http://localhost:8787/dashboard](http://localhost:8787/dashboard) . "Tokens saved" should climb as you use Claude Code. 2. Or look at the response headers on a request: `x-headroom-tokens-before`, `x-headroom-tokens-after`, `x-headroom-tokens-saved`. If "tokens saved" stays at 0 on large prompts, see Troubleshooting below. [Troubleshooting](https://docs.headroomlabs.ai/docs/claude-code-vertex#troubleshooting) ---------------------------------------------------------------------------------------- * **It still goes straight to Google (no savings).** Make sure `CLAUDE_CODE_USE_VERTEX=1` is exported _in the same shell_ before `headroom wrap claude`. The wrapper only switches to Vertex mode when it sees that variable. * **Wrong region / 404 from Vertex.** Confirm `CLOUD_ML_REGION` matches a region where your Claude model is enabled. `global` is supported and maps to the non-regional host. * **Auth errors.** Headroom forwards your token as-is — if `gcloud auth application-default login` (or `GOOGLE_APPLICATION_CREDENTIALS`) works for Claude Code without Headroom, it works with it. [Alternative: let Headroom talk to Vertex for you](https://docs.headroomlabs.ai/docs/claude-code-vertex#alternative-let-headroom-talk-to-vertex-for-you) --------------------------------------------------------------------------------------------------------------------------------------------------------- If you'd rather **not** run Claude Code in Vertex mode, you can have Headroom be the translator instead: Claude Code speaks plain Anthropic to Headroom, and Headroom calls Vertex on your behalf. export HEADROOM_BACKEND=litellm-vertex_ai # note the _ai suffix export HEADROOM_REGION=us-east5 export VERTEXAI_PROJECT= export GOOGLE_APPLICATION_CREDENTIALS=/path/sa.json # or gcloud ADC export ANTHROPIC_API_KEY=placeholder # Claude Code needs *a* key to start headroom wrap claude --backend litellm-vertex_ai --region us-east5 The native Vertex-mode flow above is recommended — it keeps your existing GCP auth and has the smallest moving parts. Use this alternative only if you can't set `CLAUDE_CODE_USE_VERTEX`. [Notes](https://docs.headroomlabs.ai/docs/claude-code-vertex#notes) -------------------------------------------------------------------- * Pick a Claude model that is enabled in your Vertex project/region (e.g. `claude-sonnet-4-6`, `claude-haiku-4-5`). * Streaming, tool use, and prompt caching all work through Headroom. * Want to point at a private Vertex gateway instead of Google's host? Start the proxy with `--vertex-api-url https://your-gateway` and Headroom will forward there. [LiteLLM\ \ Add Headroom compression to LiteLLM with a single callback. Works with all 100+ supported providers.](https://docs.headroomlabs.ai/docs/litellm) [Claude Code on Azure AI Foundry\ \ Run Claude Code against Claude models on Azure AI Foundry, with Headroom compressing your prompts — fewer input tokens, same answers, your own Azure credentials.](https://docs.headroomlabs.ai/docs/claude-code-azure-foundry) ### On this page [What you get](https://docs.headroomlabs.ai/docs/claude-code-vertex#what-you-get) [Before you start](https://docs.headroomlabs.ai/docs/claude-code-vertex#before-you-start) [Run it (one command)](https://docs.headroomlabs.ai/docs/claude-code-vertex#run-it-one-command) [How it works](https://docs.headroomlabs.ai/docs/claude-code-vertex#how-it-works) [Check that compression is working](https://docs.headroomlabs.ai/docs/claude-code-vertex#check-that-compression-is-working) [Troubleshooting](https://docs.headroomlabs.ai/docs/claude-code-vertex#troubleshooting) [Alternative: let Headroom talk to Vertex for you](https://docs.headroomlabs.ai/docs/claude-code-vertex#alternative-let-headroom-talk-to-vertex-for-you) [Notes](https://docs.headroomlabs.ai/docs/claude-code-vertex#notes) --- # Code Compression | Headroom Code Compression ================ AST-aware compression that preserves imports, signatures, and types while compressing function bodies. Powered by tree-sitter. Copy MarkdownOpen Headroom's CodeAwareCompressor uses tree-sitter to parse source code into an AST, then selectively compresses function bodies while preserving the structural elements that LLMs need -- imports, signatures, type annotations, and error handlers. [Why AST-Aware Compression?](https://docs.headroomlabs.ai/docs/code-compression#why-ast-aware-compression) ----------------------------------------------------------------------------------------------------------- Naive truncation breaks code. Cutting a function in half leaves invalid syntax that confuses the LLM. CodeAwareCompressor guarantees: * **Syntax validity** -- output always parses correctly * **Structural preservation** -- imports, signatures, types, decorators are kept intact * **Lightweight** -- ~50MB of tree-sitter parsers, loaded lazily and cached [Supported Languages](https://docs.headroomlabs.ai/docs/code-compression#supported-languages) ---------------------------------------------------------------------------------------------- | Tier | Languages | Support Level | | --- | --- | --- | | Tier 1 | Python, JavaScript, TypeScript | Full AST analysis | | Tier 2 | Go, Rust, Java, C, C++, C#, PHP | Function body compression | [What Gets Preserved vs Compressed](https://docs.headroomlabs.ai/docs/code-compression#what-gets-preserved-vs-compressed) -------------------------------------------------------------------------------------------------------------------------- **Always preserved:** * Import statements * Function and method signatures * Class definitions * Type annotations * Decorators **Compressed:** * Function bodies (implementations) * Comments (unless configured to preserve) * Verbose docstrings (configurable: full, first line, or removed) [Example](https://docs.headroomlabs.ai/docs/code-compression#example) ---------------------------------------------------------------------- from headroom.transforms import CodeAwareCompressor, CodeCompressorConfig # min_tokens_for_compression defaults to 100; a snippet this small needs an # explicit override or it passes through unmodified (see "Skip small content" below). compressor = CodeAwareCompressor(CodeCompressorConfig(min_tokens_for_compression=10)) code = ''' import os from typing import List def process_items(items: List[str]) -> List[str]: """Process a list of items.""" results = [] for item in items: if not item: continue processed = item.strip().lower() results.append(processed) return results ''' result = compressor.compress(code, language="python") print(result.compressed) # import os # from typing import List # # def process_items(items: List[str]) -> List[str]: # """Process a list of items.""" # results = [] # # [6 lines omitted] # pass # # [32 tokens compressed. Retrieve more: hash=. Expires in 5m.] print(f"Compression: {result.compression_ratio:.0%}") # ~57% print(f"Syntax valid: {result.syntax_valid}") # True [Configuration](https://docs.headroomlabs.ai/docs/code-compression#configuration) ---------------------------------------------------------------------------------- from headroom.transforms import CodeAwareCompressor, CodeCompressorConfig, DocstringMode config = CodeCompressorConfig( preserve_imports=True, # Always keep imports preserve_signatures=True, # Always keep function signatures preserve_type_annotations=True, # Keep type hints preserve_decorators=True, # Keep decorators docstring_mode=DocstringMode.FIRST_LINE, # FULL, FIRST_LINE, REMOVE target_compression_rate=0.2, # Keep 20% of tokens max_body_lines=5, # Lines to keep per function body min_tokens_for_compression=100, # Skip small content language_hint=None, # Auto-detect if None fallback_to_kompress=True, # Use Kompress for unknown langs ) compressor = CodeAwareCompressor(config) result = compressor.compress(code) ### [Configuration Options](https://docs.headroomlabs.ai/docs/code-compression#configuration-options) | Option | Default | Description | | --- | --- | --- | | `preserve_imports` | `True` | Keep all import statements | | `preserve_signatures` | `True` | Keep function/method signatures | | `preserve_type_annotations` | `True` | Keep type hints | | `preserve_decorators` | `True` | Keep decorators | | `docstring_mode` | `FIRST_LINE` | How to handle docstrings: `FULL`, `FIRST_LINE`, `REMOVE` | | `target_compression_rate` | `0.2` | Fraction of tokens to keep (0.2 = keep 20%) | | `max_body_lines` | `5` | Max lines to keep per function body | | `min_tokens_for_compression` | `100` | Skip files smaller than this | | `language_hint` | `None` | Override language detection | | `fallback_to_kompress` | `True` | Use Kompress for unsupported languages | [Before and After](https://docs.headroomlabs.ai/docs/code-compression#before-and-after) ---------------------------------------------------------------------------------------- # Before (full source file) def process_data(items: List[str]) -> Dict[str, int]: """Process items and count occurrences.""" result = {} for item in items: item = item.strip().lower() if item in result: result[item] += 1 else: result[item] = 1 return result # After (signature preserved, body compressed) def process_data(items: List[str]) -> Dict[str, int]: """Process items and count occurrences.""" result = {} for item in items: # ... (5 lines compressed) pass The LLM sees the function's purpose, its input/output types, and the general approach -- enough to reason about the code without needing every implementation line. [Installation](https://docs.headroomlabs.ai/docs/code-compression#installation) -------------------------------------------------------------------------------- # Install tree-sitter language pack pip install "headroom-ai[code]" [Memory Management](https://docs.headroomlabs.ai/docs/code-compression#memory-management) ------------------------------------------------------------------------------------------ Tree-sitter parsers are lazy-loaded and cached. You can free memory when done: from headroom.transforms import is_tree_sitter_available from headroom.transforms.code_compressor import unload_tree_sitter # Check if tree-sitter is installed print(is_tree_sitter_available()) # True # Free memory when done unload_tree_sitter() [Performance](https://docs.headroomlabs.ai/docs/code-compression#performance) ------------------------------------------------------------------------------ | Metric | Value | | --- | --- | | Compression | 40-70% token reduction | | Speed | ~10-50ms per file | | Memory | ~50MB (tree-sitter parsers) | | Syntax validity | Guaranteed | Routing differs between the proxy and the library API ContentRouter always _detects_ source code automatically, but whether it routes that code to CodeAwareCompressor depends on the entry point. `headroom proxy` enables code-aware routing by default (`HEADROOM_CODE_AWARE_ENABLED` defaults to on; disable with `--no-code-aware` or `HEADROOM_CODE_AWARE_ENABLED=0`). The one-function `compress()` API and `HeadroomClient`, however, build `ContentRouterConfig` with its own default, `enable_code_aware=False` -- there, detected code falls back to Kompress unless you construct your own pipeline with `ContentRouterConfig(enable_code_aware=True)`. Direct usage of `CodeAwareCompressor`, as shown on this page, is unaffected either way. See [How Compression Works](https://docs.headroomlabs.ai/docs/how-compression-works#content-type-detection) . [SmartCrusher\ \ Statistical JSON and array compression that keeps important items and drops the rest.](https://docs.headroomlabs.ai/docs/smart-crusher) [Image Compression\ \ ML-powered image compression that reduces vision model token usage by 40-90% while maintaining answer accuracy.](https://docs.headroomlabs.ai/docs/image-compression) ### On this page [Why AST-Aware Compression?](https://docs.headroomlabs.ai/docs/code-compression#why-ast-aware-compression) [Supported Languages](https://docs.headroomlabs.ai/docs/code-compression#supported-languages) [What Gets Preserved vs Compressed](https://docs.headroomlabs.ai/docs/code-compression#what-gets-preserved-vs-compressed) [Example](https://docs.headroomlabs.ai/docs/code-compression#example) [Configuration](https://docs.headroomlabs.ai/docs/code-compression#configuration) [Configuration Options](https://docs.headroomlabs.ai/docs/code-compression#configuration-options) [Before and After](https://docs.headroomlabs.ai/docs/code-compression#before-and-after) [Installation](https://docs.headroomlabs.ai/docs/code-compression#installation) [Memory Management](https://docs.headroomlabs.ai/docs/code-compression#memory-management) [Performance](https://docs.headroomlabs.ai/docs/code-compression#performance) --- # Recover Codex State | Headroom Recover Codex State =================== Merge sessions and configuration left in a temporary Headroom Codex home back into the durable Codex home. Copy MarkdownOpen Older versions of `headroom wrap codex` could run Codex with a temporary `CODEX_HOME` named `headroom-codex-home-*`. Chats and configuration created during that wrapped session stayed in the temporary home, so they disappeared from the normal Codex history after the wrapper exited. This recovery migrates the retained state back into the durable Codex home without deleting either source. This procedure can recover temporary homes that still exist and pinned sources retained by an interrupted recovery. A temporary home that was already deleted cannot be reconstructed unless one of those retained copies exists. Headroom reports deleted temporary homes still referenced by the Codex thread database. It does not treat paths pasted into prompts or error messages as recovery sources. See [issue #2159](https://github.com/headroomlabs-ai/headroom/issues/2159) for the regression details. [Before you recover](https://docs.headroomlabs.ai/docs/codex-recovery#before-you-recover) ------------------------------------------------------------------------------------------ Close Codex and any process using the temporary or durable Codex homes. Recovery detects changes made while creating each backup and aborts, but quiet homes are required for a consistent migration. The target defaults to `$CODEX_HOME` when it is set, otherwise `~/.codex`. Check that this is the durable home you normally use before confirming the migration. [Automatic recovery](https://docs.headroomlabs.ai/docs/codex-recovery#automatic-recovery) ------------------------------------------------------------------------------------------ The first interactive `headroom wrap codex` with the fixed version searches Python's temporary directory, `$TMPDIR`, `/tmp`, `/private/tmp`, and the macOS `/private/var/folders/*/*/T` roots for non-empty `headroom-codex-home-*` directories. If it finds any, it lists them and offers to back up and recover them before Codex starts. Declining the prompt changes nothing. You can run the manual command later. [Manual recovery](https://docs.headroomlabs.ai/docs/codex-recovery#manual-recovery) ------------------------------------------------------------------------------------ Let Headroom discover retained temporary homes and `source-pinned/` copies left by interrupted or failed recovery attempts, then preview the migration: headroom recover codex If no recoverable copy remains, the command audits the durable Codex thread database, rollout files, and `history.jsonl`. It reports indexed active and archived chats, surviving rollouts missing from the thread index, and history-only records whose rollouts no longer exist. History-only prompt text cannot reconstruct a full chat transcript. Codex filters the default resume picker by the current working directory. Run `codex resume --all` yourself to display indexed chats from every working directory. Headroom does not launch Codex during recovery. To select a known temporary home and an explicit durable target: headroom recover codex \ --source /path/to/headroom-codex-home-12345 \ --target "${CODEX_HOME:-$HOME/.codex}" Repeat `--source` to merge multiple homes. Headroom shows the target and every source before asking for confirmation. Use `--yes` only in automation where those paths have already been reviewed. [What gets migrated](https://docs.headroomlabs.ai/docs/codex-recovery#what-gets-migrated) ------------------------------------------------------------------------------------------ When a source used the localhost `headroom` model provider injected by the old wrapper, recovery rewrites that provider in both SQLite thread rows and rollout `session_meta` records to the active target provider. This also repairs newer target records left by an earlier broken recovery. A user-defined remote provider named `headroom` is preserved. * `history.jsonl` and other JSONL indexes are combined without duplicate records. Malformed input is excluded from the result and copied to the backup quarantine. * Session rollouts under `sessions/` and `archived_sessions/` keep the newer file when the same path exists in both homes. * SQLite databases are merged only when their tables, indexes, triggers, views, and migration checksums are compatible. Primary-key conflicts keep rows from the newer database. Recovered thread rows are rewritten to the durable rollout path, including rows restored from a pinned source after the original temporary home was deleted. * `config.toml` tables are merged recursively. Values from the newer config win, and localhost Headroom routing injected by the old wrapper is removed. A user-defined remote provider is preserved. * Other files, including credentials and user settings, keep the newer copy. The durable target wins equal modification-time ties, and missing source files never delete target files. Sockets, lock files, SQLite journals, FIFOs, and other runtime-only artifacts are recorded but not copied. [Backups and rollback](https://docs.headroomlabs.ai/docs/codex-recovery#backups-and-rollback) ---------------------------------------------------------------------------------------------- Every source is pinned before migration. The existing durable home, including its current Codex configuration, is backed up before any merge begins. Backups are owner-only and retained next to the target: /.headroom-codex-recovery// ├── source-pinned/ ├── target-before/ ├── target-failed/ # only when a merge is rolled back ├── manifest.json └── quarantine/ # only when malformed input is found `target-before/` is absent when the target did not exist before recovery. `manifest.json` records copied, merged, quarantined, and skipped paths. If configuration parsing, SQLite schema validation, integrity checks, foreign-key checks, or filesystem writes fail, Headroom atomically renames the failed target to `target-failed/` before restoring `target-before/`. This avoids recursive-deletion races with SQLite runtime files and preserves the failed merge for inspection. When the target did not exist before recovery, the partial target is retained only as `target-failed/`. The pinned source and recovery backup remain available for inspection. Recovery also refuses overlapping source and target paths, target symlink traversal, and a source that changes while it is being pinned. [After recovery](https://docs.headroomlabs.ai/docs/codex-recovery#after-recovery) ---------------------------------------------------------------------------------- The command prints the retained backup path after each successful source merge. Review its `manifest.json`, then start Codex normally and confirm that the recovered chats and settings appear. Keep the backup until you have verified the durable history. Fixed versions of `headroom wrap codex` run Codex against the durable home and apply proxy routing only to the launched process, so new wrapped sessions remain visible after Headroom exits. [Configuration\ \ All configuration options for the Headroom Python and TypeScript SDKs, proxy server, and per-request overrides.](https://docs.headroomlabs.ai/docs/configuration) [Pipeline Extensions\ \ Write a request-normalization extension for a quirky upstream provider, and route requests to different upstream bases per request with x-headroom-base-url.](https://docs.headroomlabs.ai/docs/pipeline-extensions) ### On this page [Before you recover](https://docs.headroomlabs.ai/docs/codex-recovery#before-you-recover) [Automatic recovery](https://docs.headroomlabs.ai/docs/codex-recovery#automatic-recovery) [Manual recovery](https://docs.headroomlabs.ai/docs/codex-recovery#manual-recovery) [What gets migrated](https://docs.headroomlabs.ai/docs/codex-recovery#what-gets-migrated) [Backups and rollback](https://docs.headroomlabs.ai/docs/codex-recovery#backups-and-rollback) [After recovery](https://docs.headroomlabs.ai/docs/codex-recovery#after-recovery) --- # Configuration | Headroom Configuration ============= All configuration options for the Headroom Python and TypeScript SDKs, proxy server, and per-request overrides. Copy MarkdownOpen Headroom can be configured via the SDK constructor, proxy command line, environment variables, or per-request overrides. [Runtime Rollout Channels](https://docs.headroomlabs.ai/docs/configuration#runtime-rollout-channels) ----------------------------------------------------------------------------------------------------- Headroom uses rollout channels to control which behaviors an already-installed artifact may expose. They do not select a package or released version. | Variable | Default | Purpose | | --- | --- | --- | | `HEADROOM_ROLLOUT_CHANNEL` | `stable` | Selects `stable`, `beta`, `canary`, or `dev`. | | `HEADROOM_FEATURES` | unset | Comma-separated feature names to request explicitly. | | `HEADROOM_DISABLE_FEATURES` | unset | Comma-separated feature names to force off. Disable wins over every enable path. | | `HEADROOM_UNSAFE_ALLOW_UNSTABLE_FEATURES` | unset | Break-glass override for emergency mitigation only. | See [Runtime Rollouts](https://docs.headroomlabs.ai/docs/runtime-rollouts) for policy, provenance, and contributor rules. If Codex history disappeared after using an older wrapper, see [Recover Codex State](https://docs.headroomlabs.ai/docs/codex-recovery) before wrapping Codex again. [SDK Modes (`default_mode` / `headroom_mode`)](https://docs.headroomlabs.ai/docs/configuration#sdk-modes-default_mode--headroom_mode) -------------------------------------------------------------------------------------------------------------------------------------- These modes apply to SDK usage via `HeadroomClient(default_mode=...)` or per-request `headroom_mode=...`. They are **not** the same as the proxy `--mode` flag. | Mode | Behavior | Use Case | | --- | --- | --- | | `audit` | Observes and logs, no modifications | Production monitoring, baseline measurement | | `optimize` | Applies safe, deterministic transforms | Production optimization | | `simulate` | Returns plan without API call | Testing, cost estimation | > **Proxy `--mode` is a separate axis**: `--mode cache` (the default) freezes prior turns so the provider's prefix cache is never busted, which is where the savings come from on a real agent workload. `--mode token` maximizes visible per-request compression at the cost of cache stability; prefer `cache` unless you are specifically diagnosing compression numbers. The proxy does not accept `audit`, `optimize`, or `simulate`. [SDK Configuration](https://docs.headroomlabs.ai/docs/configuration#sdk-configuration) --------------------------------------------------------------------------------------- TypeScriptPython import { HeadroomClient } from 'headroom-ai'; // Reads from HEADROOM_BASE_URL and HEADROOM_API_KEY automatically const client = new HeadroomClient(); // Or configure explicitly const explicit = new HeadroomClient({ baseUrl: 'http://localhost:8787', apiKey: 'your-api-key', timeout: 30_000, fallback: true, retries: 2, }); from headroom import HeadroomClient, OpenAIProvider from openai import OpenAI client = HeadroomClient( original_client=OpenAI(), provider=OpenAIProvider(), # Mode: "audit" (observe only) or "optimize" (apply transforms) default_mode="optimize", # Enable provider-specific cache optimization enable_cache_optimizer=True, # Enable query-level semantic caching enable_semantic_cache=False, # Override default context limits per model model_context_limits={ "gpt-4o": 128000, "gpt-4o-mini": 128000, }, # Database location (defaults to temp directory) # store_url="sqlite:////absolute/path/to/headroom.db", ) [Per-Request Overrides](https://docs.headroomlabs.ai/docs/configuration#per-request-overrides) ----------------------------------------------------------------------------------------------- Override configuration for individual requests: TypeScriptPython import { compress } from 'headroom-ai'; const result = await compress(messages, { model: 'gpt-4o', tokenBudget: 100_000, timeout: 15_000, }); response = client.chat.completions.create( model="gpt-4o", messages=[...], # Override mode for this request headroom_mode="audit", # Reserve more tokens for output headroom_output_buffer_tokens=8000, # Keep last N turns (don't compress) headroom_keep_turns=5, # Skip compression for specific tools headroom_tool_profiles={ "important_tool": {"skip_compression": True} }, ) ### [Proxy upstream override (`x-headroom-base-url`)](https://docs.headroomlabs.ai/docs/configuration#proxy-upstream-override-x-headroom-base-url) When using the proxy, send the `x-headroom-base-url` request header to route a single request to a different upstream instead of the configured provider URL. This lets a client that speaks a provider's wire format authenticate against a compatible gateway (for example an OpenAI-compatible endpoint, or an Anthropic-Messages gateway such as OpenCode Zen) without changing the proxy configuration. The header is honored by the OpenAI-compatible routes, the Anthropic Messages route (`POST /v1/messages`), and the generic passthrough route. The proxy forwards the request to `` + the original request path (e.g. `/v1/messages`). An empty or whitespace-only value is ignored and the configured upstream is used. curl http://127.0.0.1:8787/v1/messages \ -H "content-type: application/json" \ -H "anthropic-version: 2023-06-01" \ -H "x-headroom-base-url: https://opencode.ai/zen/go" \ -H "x-api-key: " \ -d '{"model":"glm-5.2","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}' When `HEADROOM_STRIP_INTERNAL_HEADERS` is `enabled` (the default), the proxy reads this header for routing and then strips it before forwarding upstream. #### [Configured secret headers are not sent to arbitrary upstreams](https://docs.headroomlabs.ai/docs/configuration#configured-secret-headers-are-not-sent-to-arbitrary-upstreams) `ANTHROPIC_TARGET_API_HEADERS` / `OPENAI_TARGET_API_HEADERS` hold operator secrets. Because `x-headroom-base-url` is chosen by the _client_, those headers are only attached when the destination is one the operator designated: * a host in the configured provider targets (`ANTHROPIC_TARGET_API_URL`, `OPENAI_TARGET_API_URL`, and the Gemini/Vertex/Cloud Code equivalents), or * a host listed in `HEADROOM_UPSTREAM_ALLOWED_HOSTS` (comma-separated). A request to any other upstream is **still proxied** — it just does not carry your configured headers, and the proxy logs `upstream_extra_headers_withheld host=` once per host. If you route to a gateway via this header and need your configured headers to reach it, add its host to `HEADROOM_UPSTREAM_ALLOWED_HOSTS`: export HEADROOM_UPSTREAM_ALLOWED_HOSTS="gateway.internal,api.example-gateway.ai" Matching is on the parsed hostname and is exact — no wildcards — so `api.anthropic.com.evil.example` and `https://api.anthropic.com@evil.example` do not match `api.anthropic.com`. [SmartCrusher Configuration](https://docs.headroomlabs.ai/docs/configuration#smartcrusher-configuration) --------------------------------------------------------------------------------------------------------- Fine-tune JSON compression behavior: from headroom.transforms import SmartCrusherConfig config = SmartCrusherConfig( # Maximum items to keep after compression max_items_after_crush=15, # Minimum tokens before applying compression min_tokens_to_crush=200, # Fraction of items always kept from the start/end first_fraction=0.3, last_fraction=0.15, # Variance threshold for statistical analysis variance_threshold=2.0, ) [CacheAligner Configuration](https://docs.headroomlabs.ai/docs/configuration#cachealigner-configuration) --------------------------------------------------------------------------------------------------------- Control prefix stabilization for provider cache hit rates: from headroom import CacheAlignerConfig config = CacheAlignerConfig( # Enable/disable cache alignment enabled=True, # Use custom regex patterns instead of the built-in dynamic-content # detector (use_dynamic_detector=False activates date_patterns) use_dynamic_detector=False, date_patterns=[\ r"Today is \w+ \d+, \d{4}",\ r"Current time: .*",\ ], ) [Context Window Management](https://docs.headroomlabs.ai/docs/configuration#context-window-management) ------------------------------------------------------------------------------------------------------- Context management is now automatic. Use per-request overrides to control behavior: response = client.chat.completions.create( model="gpt-4o", messages=messages, # Reserve tokens for model output headroom_output_buffer_tokens=4000, # Keep last N turns uncompressed headroom_keep_turns=3, ) The `RollingWindowConfig`, `IntelligentContextConfig`, and `ScoringWeights` classes are no longer part of Headroom. Context management now happens automatically inside the pipeline (live-zone-only compression). ### [Claude 1M context window (`headroom wrap claude --1m`)](https://docs.headroomlabs.ai/docs/configuration#claude-1m-context-window-headroom-wrap-claude---1m) `headroom wrap claude --1m` opts a Claude Code session into Anthropic's 1M-token context window by selecting a `[1m]`\-suffixed model id, which makes Claude Code send the `context-1m` beta header. The model that `--1m` targets is resolved in this order: 1. an explicit `--model` / `ANTHROPIC_MODEL` value (used as-is, with a `[1m]` suffix appended when missing), 2. otherwise `HEADROOM_1M_MODEL`, when set, 3. otherwise the built-in default (currently `claude-opus-5`). Set `HEADROOM_1M_MODEL` to point `--1m` at a specific model without pinning `ANTHROPIC_MODEL` globally, so the default can follow a new Opus generation without a code change: # Route --1m at a specific model for this shell / session export HEADROOM_1M_MODEL=claude-opus-5 headroom wrap claude --1m `HEADROOM_1M_MODEL` is a fallback only: an explicit `--model` or `ANTHROPIC_MODEL` always wins. The value may be given with or without the `[1m]` suffix; both `claude-opus-5` and `claude-opus-5[1m]` are accepted, and the suffix is added when absent. [Pipeline Extensions](https://docs.headroomlabs.ai/docs/configuration#pipeline-extensions) ------------------------------------------------------------------------------------------- Use a `headroom.pipeline_extension` entry point when you need to normalize or annotate requests before they leave Headroom. The `PRE_SEND` stage is the right place for provider-specific request cleanup, such as turning `content: null` into `content: ""` for upstreams that reject OpenAI-spec tool-call messages. from headroom.pipeline import PipelineEvent, PipelineStage class NormalizeNullContent: def on_pipeline_event(self, event: PipelineEvent) -> PipelineEvent: if event.stage is not PipelineStage.PRE_SEND or not event.messages: return event for message in event.messages: if ( message.get("role") == "assistant" and message.get("content") is None and message.get("tool_calls") ): message["content"] = "" return event Register it in `pyproject.toml`: [project.entry-points."headroom.pipeline_extension"] normalize_null_content = "my_pkg.normalize:NormalizeNullContent" If the upstream base URL itself must vary per request, use the `x-headroom-base-url` override header in addition to the normalization hook. [Proxy Configuration](https://docs.headroomlabs.ai/docs/configuration#proxy-configuration) ------------------------------------------------------------------------------------------- ### [Command Line Options](https://docs.headroomlabs.ai/docs/configuration#command-line-options) headroom proxy \ --port 8787 \ # Port to listen on --host 0.0.0.0 \ # Host to bind to --mode cache \ # default; freezes prior turns for prefix-cache stability --budget 10.00 \ # Daily budget limit in USD --log-file headroom.jsonl # Log file path ### [Feature Flags](https://docs.headroomlabs.ai/docs/configuration#feature-flags) # Disable optimization (passthrough mode) headroom proxy --no-optimize # Disable semantic caching headroom proxy --no-cache # Preserve provider prefix-cache stability instead of maximizing token removal headroom proxy --mode cache # Enable memory and live learning headroom proxy --memory headroom proxy --learn --min-evidence 3 [Environment Variables](https://docs.headroomlabs.ai/docs/configuration#environment-variables) ----------------------------------------------------------------------------------------------- | Variable | Description | Default | | --- | --- | --- | | `HEADROOM_HOST` | Proxy bind host | `127.0.0.1` | | `HEADROOM_PORT` | Proxy bind port | `8787` | | `HEADROOM_MODE` | Proxy optimization mode: `token` or `cache` | `cache` | | `HEADROOM_WORKERS` | Uvicorn worker count | `1` | | `HEADROOM_LIMIT_CONCURRENCY` | Maximum concurrent connections before 503 | `1000` | | `HEADROOM_MAX_CONNECTIONS` | Maximum upstream HTTP connections | `500` | | `HEADROOM_MAX_KEEPALIVE` | Maximum upstream keep-alive connections | `100` | | `HEADROOM_KEEPALIVE_EXPIRY` | Seconds an idle upstream keep-alive connection is kept open | `90` | | `HEADROOM_HTTP_PROXY` | HTTP proxy URL for upstream provider requests only; HTTPS provider APIs use CONNECT | \-- | | `HEADROOM_BUDGET` | Daily budget limit in USD | \-- | | `HEADROOM_TELEMETRY` | Set to `on` for **local-only** usage stats (powers your own `/stats` and dashboard; nothing is sent externally) | `off` | | `HEADROOM_STATELESS` | Set to `true` to disable filesystem writes | `false` | | `HEADROOM_MODEL_LIMITS` | Custom model config (JSON string or file path) | \-- | | `HEADROOM_LOSSLESS_COMPACTION` | Set to `0` to disable the byte-reversible lossless folds (grep heading, log run-collapse, diff/config folds) while leaving the rest of the compression pipeline active. Read per request, so the proxy's `/admin/runtime-env` hot-sync applies without a restart. | `1` | | `HEADROOM_BASE_URL` | Base URL of the Headroom proxy (TypeScript SDK) | `http://localhost:8787` | | `HEADROOM_API_KEY` | Optional API key for authenticated Headroom endpoints (TypeScript SDK) | \-- | | `HEADROOM_CONFIG_DIR` | Canonical config (read-mostly) root. Derives `models.json` and per-plugin config paths when set. | `~/.headroom/config` | | `HEADROOM_WORKSPACE_DIR` | Canonical workspace (read-write state) root. Derives savings, memory DB, logs, TOIN, subscription state, and more when set. | `~/.headroom` | | `HEADROOM_SAVINGS_PATH` | Override persistent savings file location. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` | | `HEADROOM_TOIN_PATH` | Override TOIN telemetry file location. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` | | `HEADROOM_SUBSCRIPTION_STATE_PATH` | Override subscription tracker state file. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` | | `HEADROOM_PERIODIC_TOIN_STATS` | Controls periodic TOIN stats logging in long-lived proxy workers. Set to `0`, `false`, `off`, or `no` to disable the 5-minute stats loop without disabling TOIN learning or request-time feedback. | `true` | | `HEADROOM_MEMORY_INJECTION_MODE` | Memory-context routing mode: `live_zone_tail` (default) or `disabled`. The legacy `system_prompt` mode was retired by PR-A2; supplying it raises. | `live_zone_tail` | | `HEADROOM_PROXY_PYTHON_FORWARDER_MODE` | Python forwarder serialization mode. `byte_faithful` (default) forwards original request bytes verbatim when no transform mutated the body and re-serializes canonically only when needed — keeps Anthropic prompt-cache hit-rate intact. `legacy_json_kwarg` is an explicit operator opt-in for emergency rollback to the historical `httpx ... json=body` behavior. NOT a fallback — only flip on explicit operator decision. | `byte_faithful` | | `HEADROOM_STRIP_INTERNAL_HEADERS` | Python proxy: whether to strip internal `x-headroom-*` request headers (e.g. `x-headroom-bypass`, `x-headroom-mode`, `x-headroom-user-id`, `x-headroom-stack`, `x-headroom-base-url`) before every upstream forwarder call (PR-A5, fixes P5-49). `enabled` (default) stops fingerprinting / leakage. `disabled` is an explicit operator opt-in for diagnostic shadow tracing — NOT a fallback. Inbound reads of these headers (bypass gating, memory user-id resolution) are unaffected because they read `request.headers` directly. | `enabled` | | `HEADROOM_PROXY_STRIP_INTERNAL_HEADERS` | Rust proxy: same policy as `HEADROOM_STRIP_INTERNAL_HEADERS` but for the Rust transparent proxy. Stripping happens inside `build_forward_request_headers` so both HTTP and WebSocket upstream calls are gated by one flag. `enabled` default; `disabled` operator opt-in for diagnostic shadow tracing. Response-side `X-Headroom-*` injection (e.g. `x-headroom-tokens-saved`) is unrelated and stays. | `enabled` | | `HEADROOM_EMBEDDER_RUNTIME` | Set to `pytorch_mps` to run the memory embedder via the torch sentence-transformers backend on the Apple GPU (MPS). Only engages when Apple MPS is actually available; otherwise it logs a warning and uses the existing default embedder selection path. `pytorch_mps` is the only accepted value. Requires the `[pytorch-mps]` extra. See [Memory](https://docs.headroomlabs.ai/docs/memory#embedding-runtime--gpu-offload-apple-silicon)
. | default embedder selection | | `HEADROOM_KOMPRESS_BACKEND` | Selects the Kompress (ModernBERT) inference backend, separate from `HEADROOM_EMBEDDER_RUNTIME` above. `auto` tries ONNX CPU then falls back to PyTorch; `onnx`/`onnx_cpu` and `onnx_coreml` force ONNX Runtime (CPU or CoreML); `pytorch` and `pytorch_mps` force PyTorch (`pytorch_mps` targets Apple's GPU, with `mps`/`torch_mps` accepted as aliases). | `auto` | | `ORT_DYLIB_PATH` | Path to the ONNX Runtime shared library loaded by the Rust core (magika detection, fastembed embeddings), which loads ORT dynamically on every platform. Auto-pinned at `import headroom` to the library inside the `onnxruntime` pip package (`onnxruntime.dll` / `libonnxruntime.so*` / `libonnxruntime*.dylib`); set it yourself to override. Without a pin, ML detection degrades to the non-ONNX tiers — and on Windows the bare DLL search can resolve to the Windows ML System32 build (1.17.x on Win11 24H2+), which deadlocks ONNX session init — see [Troubleshooting](https://docs.headroomlabs.ai/docs/troubleshooting#windows-ml-content-detection-hangs-or-silently-falls-back)
. | auto-pinned | | `HEADROOM_MAGIKA_INIT_TIMEOUT_SECS` | Upper bound (integer seconds, > 0) on magika's one-time ONNX session init in the Rust detection chain. On timeout the init error is cached and detection uses the non-ML fallback tiers for the rest of the process; a warning is logged. Safety net for environments where the dylib pin above does not apply. | `5` | | `HEADROOM_REQUEST_TIMEOUT` | Request timeout in seconds | `300` | | `HEADROOM_BETA_HEADER_STICKY` | Controls per-session `anthropic-beta` / `OpenAI-Beta` re-echo. `enabled` (default): the proxy unions beta tokens across turns within a session — if the client sends a token in turn N and omits it in turn N+1, the proxy re-injects it to preserve prefix-cache stability. `disabled`: the client's value is forwarded verbatim with no accumulation. Any other value raises at request time. See [Session Beta Header Tracking](https://docs.headroomlabs.ai/docs/configuration#session-beta-header-tracking)
. | `enabled` | | `HEADROOM_BETA_TRACKER_MAX_SESSIONS` | LRU capacity of the in-memory session beta tracker. Once full, the oldest session entry is evicted. | `1000` | | `HEADROOM_PROXY_BETA_HEADER_STICKY` | Rust proxy: same per-conversation beta-token union as `HEADROOM_BETA_HEADER_STICKY`, applied to `anthropic-beta` / `openai-beta` on the intercepted `/v1/messages`, `/v1/chat/completions`, and `/v1/responses` routes. Requires the compression interceptor (`HEADROOM_PROXY_COMPRESSION=1`) — with it off the Rust proxy is a strict byte-pipe and this flag has no effect (startup warns). Unlike the Python tracker (keyed on model + system prompt), sessions are keyed per conversation, shared with the cache-drift detector — parallel conversations never inherit each other's tokens. `enabled` default; `disabled` forwards the client value verbatim and keeps no state. Tracker capacity is fixed at 1000 sessions. | `enabled` | | `HEADROOM_MODEL_ROUTER_ENABLED` | Enable cost-aware model routing. `1`/`true`/`yes`/`on`/`enabled` turns it on and requires `HEADROOM_MODEL_ROUTES`. See [Cost-aware model routing](https://docs.headroomlabs.ai/docs/configuration#cost-aware-model-routing)
. | `off` | | `HEADROOM_MODEL_ROUTES` | JSON array of ordered routing rules for cost-aware model routing (schema below). | \-- | | `HEADROOM_THINKING_COMPACT` | Compact plain-text reasoning that models re-send every turn (Kimi/GLM/DeepSeek `reasoning_content` / inline ``): Kompress it on warm turns, drop it on cold turns. No-op for Claude/Codex/OpenAI (encrypted reasoning). See [Cold-prefix hook](https://docs.headroomlabs.ai/docs/configuration#cold-prefix-hook--reasoning-compaction)
. | `off` | | `HEADROOM_THINKING_COMPACT_KEEP_LAST` | Most-recent assistant turns whose reasoning is left intact (the active reasoning the model still uses). Only applies with `HEADROOM_THINKING_COMPACT`. | `1` | | `HEADROOM_COLD_RECOMPACT` | On a confirmed-cold turn (idle past the real cache TTL), recompact the whole prefix — cross-turn dedupe + superseded-read drop + lossless folds — instead of forwarding a byte-identical prefix to a dead cache. Warm turns unchanged. Pair with `HEADROOM_DEDUPE`. See [Cold-prefix hook](https://docs.headroomlabs.ai/docs/configuration#cold-prefix-hook--reasoning-compaction)
. | `off` | | `HEADROOM_DEDUPE` | Whole-conversation verbatim cross-turn dedup in the router (cache-safe, information-preserving via retrieval markers). Superseded-read drop + lossless folds run without it; this adds verbatim dedup. | `off` | | `HEADROOM_CACHE_TTL_LEARN` | Append per-turn cache-outcome observations (provider, model, idle, hit/miss) to `cache_ttl_observations.jsonl` for the offline `headroom-cache-ttl` learner. Observation-only (no request-behavior change); respects `HEADROOM_STATELESS`; the log is size-bounded. | `off` | | `HEADROOM_KOMPRESS_ENDPOINT` / `HEADROOM_KOMPRESS_ENDPOINT_TOKEN` | Offload ML compression (Kompress) to a remote endpoint instead of the local ONNX model — used by reasoning compaction and the router when set. | \-- | | `HEADROOM_1M_MODEL` | Fallback model that `headroom wrap claude --1m` targets when neither `--model` nor `ANTHROPIC_MODEL` is set. Accepts the id with or without the `[1m]` suffix (added when absent); an explicit `--model` / `ANTHROPIC_MODEL` always wins. See [Claude 1M context window](https://docs.headroomlabs.ai/docs/configuration#claude-1m-context-window-headroom-wrap-claude---1m)
. | `claude-opus-5` | For provider-only proxying, prefer `HEADROOM_HTTP_PROXY` over process-wide variables such as `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, or `NO_PROXY`. HTTPX reads those global variables, but Headroom also passes them through to tool executions. ### [Cold-prefix hook & reasoning compaction](https://docs.headroomlabs.ai/docs/configuration#cold-prefix-hook--reasoning-compaction) Two related optimizations target tokens that agent harnesses re-send every turn: model **reasoning** (re-billed as input on Kimi/GLM/DeepSeek and on Claude 4.6+), and the **prefix** itself once its prompt cache has lapsed. All flags below are **off by default** — the base proxy is unchanged until you opt in. **What to set for what:** | Goal | Set | Notes | | --- | --- | --- | | Shrink Kimi/GLM/DeepSeek reasoning re-sent each turn | `HEADROOM_THINKING_COMPACT=1` (+ `HEADROOM_KOMPRESS_ENDPOINT` for the best ratio) | Warm turns Kompress the reasoning; cold turns drop it. No-op for Claude/Codex/OpenAI (their reasoning is an encrypted handle, already cheap on resend). | | Recompact a dead-cache prefix instead of re-sending it whole | `HEADROOM_COLD_RECOMPACT=1` (+ `HEADROOM_DEDUPE=1`) | Fires only when the cache has genuinely lapsed. Works in both `cache` and `token` mode. | | Learn each provider's real cache TTL | `HEADROOM_CACHE_TTL_LEARN=1`, then run the `headroom-cache-ttl` estimator periodically | Sharpens cold detection for providers that don't expose a TTL (Kimi/OpenAI). | **How cold detection knows the TTL.** A wrong "cold" call would recompact a _warm_ prefix and bust the cache, so the TTL must be right: * **Claude Code:** read exactly, from the request's `cache_control.ttl` plus CC's own controls — `ENABLE_PROMPT_CACHING_1H` (1h), `FORCE_PROMPT_CACHING_5M` (5m), and `DISABLE_PROMPT_CACHING` (+ per-model `DISABLE_PROMPT_CACHING_`), which turns caching off and makes every turn a free recompaction candidate. * **Kimi / OpenAI / Codex:** they don't expose a TTL, so detection uses a conservative default until the learner (`HEADROOM_CACHE_TTL_LEARN` + the `headroom-cache-ttl` plugin) fills in the real value from observed hits/misses. **Running the estimator.** With `HEADROOM_CACHE_TTL_LEARN=1` set on the proxy, run the batch estimator periodically (cron / launchd — it is deliberately out-of-process so it can never touch the request path): headroom-cache-ttl # reads cache_ttl_observations.jsonl, writes cache_ttl_learned.json headroom-cache-ttl --dry-run # print the estimate table without writing It emits a TTL for a `provider/model` only when the observations bound it on both sides — hits at some idle prove the cache was still alive (lower bound), `ttl_expiry` misses at a larger idle prove it was dead (upper bound) — and it writes the upper end of that interval: overestimating a TTL only skips a recompaction, while underestimating one would bust a warm cache. Estimates are never pooled across models: one model's expiry says nothing about another's TTL, so a cross-model aggregate could close one model's interval with another's death evidence. Keys without death-evidence beyond their last observed hit are skipped, as is any model with no estimate of its own; both keep the conservative static default. The proxy picks up a rewritten table by mtime; no restart needed. **Can these be on by default?** * `HEADROOM_THINKING_COMPACT` — **stays opt-in.** It rewrites model inputs (reasoning the model actively uses) and depends on Kompress, so its quality / latency trade should be a deliberate choice. * `HEADROOM_COLD_RECOMPACT` — **opt-in today; a candidate to default for Claude Code** once TTL detection is field-validated. It only fires on confirmed-cold turns and recompacts losslessly, but a mis-read TTL would bust a warm cache (expensive), so it waits for confidence. For Kimi/OpenAI it should stay opt-in until the learner has data. * `HEADROOM_CACHE_TTL_LEARN` — **the safest to default on:** observation-only, a size-bounded local log, and it respects `HEADROOM_STATELESS`. Kept opt-in for now so nothing is written to disk unasked. ### [Cost-aware model routing](https://docs.headroomlabs.ai/docs/configuration#cost-aware-model-routing) Complementary to content compression, Headroom can rewrite the upstream model per request to stretch quota and control spend, for example by sending small, tool-free requests to a cheaper model. Routing is opt-in and disabled by default, so behavior is unchanged unless you configure it. Enable it with `HEADROOM_MODEL_ROUTER_ENABLED=1` and declare ordered rules in `HEADROOM_MODEL_ROUTES` (a JSON array). The router evaluates rules top to bottom and the first rule whose conditions all match wins; every decision is logged with a reason so routing stays observable. Each rule object supports: | Field | Type | Meaning | | --- | --- | --- | | `to_model` | string (required) | Model to route to when the rule matches. | | `max_input_tokens` | integer | Match only when the estimated input size is at or below this. | | `min_input_tokens` | integer | Match only when the estimated input size is at or above this. | | `require_no_tools` | boolean | Match only when the request declares no tools (a proxy for low-risk work). | | `from_models` | list of strings | Restrict the rule to these source models. Omit for any source model. | | `name` | string | Label surfaced in the decision log. | export HEADROOM_MODEL_ROUTER_ENABLED=1 export HEADROOM_MODEL_ROUTES='[\ {"name": "small-no-tools", "max_input_tokens": 4000, "require_no_tools": true,\ "from_models": ["claude-sonnet-4-6"], "to_model": "claude-haiku-4-5"}\ ]' Notes: * Input size is a fast, tokenizer-free estimate over the messages, tools, and top-level `system` prompt, meant for tier selection rather than exact accounting. * A malformed rule fails open: it is skipped (never silently widened), and the rest of the rules still apply. * Routing is skipped for byte-faithful passthrough requests (`x-headroom-bypass: true` or `x-headroom-mode: passthrough`), so those are never model-rewritten. * Routing currently applies on the Anthropic `/v1/messages` path. ### [Session Beta Header Tracking](https://docs.headroomlabs.ai/docs/configuration#session-beta-header-tracking) When running as a proxy, Headroom maintains a per-session union of `anthropic-beta` (and `OpenAI-Beta`) tokens via `SessionBetaTracker`. The session key is derived from the `x-headroom-session-id` header if present, otherwise from `md5(model + system_prompt[:500])[:16]` — stable across turns of the same conversation. **Why:** clients such as Claude Code and Codex CLI may drop a beta token between consecutive turns. Because `anthropic-beta` is part of the request bytes that determine the upstream prefix-cache key, a dropped token would bust the cache mid-conversation. The tracker re-injects any token seen earlier in the session so the cache key stays stable. **Trade-off:** once the proxy has seen a beta token in a session it will continue re-sending it for the rest of that session, even if the client stops including it. Stopping the token on the client side alone is not sufficient — the proxy re-injects it. Set `HEADROOM_BETA_HEADER_STICKY=disabled` to pass the client's `anthropic-beta` value verbatim and bypass this accumulation. # Disable sticky beta re-echo export HEADROOM_BETA_HEADER_STICKY=disabled headroom proxy ... Note: disabling sticky mode may reduce prefix-cache hit rates for clients that legitimately drop-and-re-add beta tokens across turns. ### [Filesystem Contract](https://docs.headroomlabs.ai/docs/configuration#filesystem-contract) Headroom resolves every on-disk resource through a two-root model (`HEADROOM_CONFIG_DIR` + `HEADROOM_WORKSPACE_DIR`) with additive precedence rules: explicit argument > per-resource env var > derived from canonical root > default. Every legacy env var continues to work unchanged. See the **[Filesystem Contract](https://docs.headroomlabs.ai/docs/filesystem-contract) ** page for the full bucket table, plugin-author guidance, and the Docker naming overlap note (`HEADROOM_WORKSPACE` is _not_ the same as `HEADROOM_WORKSPACE_DIR`). [Custom Model Configuration](https://docs.headroomlabs.ai/docs/configuration#custom-model-configuration) --------------------------------------------------------------------------------------------------------- Configure context limits and pricing for new or custom models: { "anthropic": { "context_limits": { "claude-4-opus-20250301": 200000, "claude-custom-finetune": 128000 }, "pricing": { "claude-4-opus-20250301": { "input": 15.00, "output": 75.00, "cached_input": 1.50 } } }, "openai": { "context_limits": { "gpt-5": 256000, "ft:gpt-4o:my-org": 128000 } } } Save as `${HEADROOM_CONFIG_DIR}/models.json` (defaults to `~/.headroom/config/models.json`), or set `HEADROOM_MODEL_LIMITS` to a JSON string or file path. Installs that still have `~/.headroom/models.json` (the legacy location) continue to work. Settings are resolved in this order (later overrides earlier): 1. Built-in defaults 2. `${HEADROOM_CONFIG_DIR}/models.json` (new canonical location); falls back to `~/.headroom/models.json` (legacy) when the canonical file is absent 3. `HEADROOM_MODEL_LIMITS` environment variable 4. SDK constructor arguments ### [Pattern-Based Inference](https://docs.headroomlabs.ai/docs/configuration#pattern-based-inference) Unknown models are automatically inferred from naming patterns: | Pattern | Inferred Settings | | --- | --- | | `*opus*` | 200K context, Opus-tier pricing | | `*sonnet*` | 200K context, Sonnet-tier pricing | | `*haiku*` | 200K context, Haiku-tier pricing | | `gpt-4o*` | 128K context, GPT-4o pricing | | `o1*`, `o3*` | 200K context, reasoning model pricing | [Provider-Specific Settings](https://docs.headroomlabs.ai/docs/configuration#provider-specific-settings) --------------------------------------------------------------------------------------------------------- Provider classes configure token counting and context/pricing overrides, not cache toggles -- provider-specific cache optimization (Anthropic `cache_control` breakpoints, OpenAI prefix stabilization, Google `CachedContent`) is controlled by `enable_cache_optimizer` on `HeadroomClient` (see [SDK Configuration](https://docs.headroomlabs.ai/docs/configuration#sdk-configuration) above), which auto-detects the right strategy from the provider you pass in. OpenAIAnthropicGoogle from headroom import OpenAIProvider provider = OpenAIProvider( context_limits={"gpt-4o": 128000}, ) from headroom import AnthropicProvider from anthropic import Anthropic provider = AnthropicProvider( # Optional: enables accurate token counting via Anthropic's Token Count API client=Anthropic(), ) from headroom.providers import GoogleProvider # Optional: pass a configured google.generativeai module for API-based # token counting via countTokens; omit for estimation-based counting. provider = GoogleProvider() [Tool Profiles](https://docs.headroomlabs.ai/docs/configuration#tool-profiles) ------------------------------------------------------------------------------- Skip or customize compression for specific tools: response = client.chat.completions.create( model="gpt-4o", messages=messages, headroom_tool_profiles={ "important_tool": {"skip_compression": True}, "search_tool": {"max_items_after_crush": 25}, }, ) [Configuration Precedence](https://docs.headroomlabs.ai/docs/configuration#configuration-precedence) ----------------------------------------------------------------------------------------------------- Settings are applied in this order (later overrides earlier): 1. Default values 2. Environment variables 3. SDK constructor arguments 4. Per-request overrides [Validation](https://docs.headroomlabs.ai/docs/configuration#validation) ------------------------------------------------------------------------- Validate your configuration at startup: result = client.validate_setup() if not result["valid"]: print("Configuration issues:") for check in ("provider", "storage", "config", "cache_optimizer"): if result[check]["error"]: print(f" - {check}: {result[check]['error']}") [MCP Tools\ \ Compression, retrieval, and stats as MCP tools for Claude Code, Cursor, and any MCP-compatible host.](https://docs.headroomlabs.ai/docs/mcp) [Recover Codex State\ \ Merge sessions and configuration left in a temporary Headroom Codex home back into the durable Codex home.](https://docs.headroomlabs.ai/docs/codex-recovery) ### On this page [Runtime Rollout Channels](https://docs.headroomlabs.ai/docs/configuration#runtime-rollout-channels) [SDK Modes (`default_mode` / `headroom_mode`)](https://docs.headroomlabs.ai/docs/configuration#sdk-modes-default_mode--headroom_mode) [SDK Configuration](https://docs.headroomlabs.ai/docs/configuration#sdk-configuration) [Per-Request Overrides](https://docs.headroomlabs.ai/docs/configuration#per-request-overrides) [Proxy upstream override (`x-headroom-base-url`)](https://docs.headroomlabs.ai/docs/configuration#proxy-upstream-override-x-headroom-base-url) [Configured secret headers are not sent to arbitrary upstreams](https://docs.headroomlabs.ai/docs/configuration#configured-secret-headers-are-not-sent-to-arbitrary-upstreams) [SmartCrusher Configuration](https://docs.headroomlabs.ai/docs/configuration#smartcrusher-configuration) [CacheAligner Configuration](https://docs.headroomlabs.ai/docs/configuration#cachealigner-configuration) [Context Window Management](https://docs.headroomlabs.ai/docs/configuration#context-window-management) [Claude 1M context window (`headroom wrap claude --1m`)](https://docs.headroomlabs.ai/docs/configuration#claude-1m-context-window-headroom-wrap-claude---1m) [Pipeline Extensions](https://docs.headroomlabs.ai/docs/configuration#pipeline-extensions) [Proxy Configuration](https://docs.headroomlabs.ai/docs/configuration#proxy-configuration) [Command Line Options](https://docs.headroomlabs.ai/docs/configuration#command-line-options) [Feature Flags](https://docs.headroomlabs.ai/docs/configuration#feature-flags) [Environment Variables](https://docs.headroomlabs.ai/docs/configuration#environment-variables) [Cold-prefix hook & reasoning compaction](https://docs.headroomlabs.ai/docs/configuration#cold-prefix-hook--reasoning-compaction) [Cost-aware model routing](https://docs.headroomlabs.ai/docs/configuration#cost-aware-model-routing) [Session Beta Header Tracking](https://docs.headroomlabs.ai/docs/configuration#session-beta-header-tracking) [Filesystem Contract](https://docs.headroomlabs.ai/docs/configuration#filesystem-contract) [Custom Model Configuration](https://docs.headroomlabs.ai/docs/configuration#custom-model-configuration) [Pattern-Based Inference](https://docs.headroomlabs.ai/docs/configuration#pattern-based-inference) [Provider-Specific Settings](https://docs.headroomlabs.ai/docs/configuration#provider-specific-settings) [Tool Profiles](https://docs.headroomlabs.ai/docs/configuration#tool-profiles) [Configuration Precedence](https://docs.headroomlabs.ai/docs/configuration#configuration-precedence) [Validation](https://docs.headroomlabs.ai/docs/configuration#validation) --- # Context Management | Headroom Context Management ================== Automatic live-zone-only context management that compresses the newest content blocks while preserving the provider cache hot zone. Copy MarkdownOpen Context management is now handled automatically inside the pipeline (live-zone-only compression). Headroom **never drops messages** from the conversation history and does not do position-based or score-based context management. [How It Works](https://docs.headroomlabs.ai/docs/context-management#how-it-works) ---------------------------------------------------------------------------------- Headroom compresses only the **newest content blocks** — the latest user message and the latest tool result / tool output. Compression is type-aware and reversible via [CCR](https://docs.headroomlabs.ai/docs/ccr) , so the LLM can retrieve the original content on demand. The **cache hot zone** — the system prompt, tool definitions, and older turns — is never mutated. Leaving the prefix untouched preserves provider prompt caching, so cache hit rates stay stable across turns. Conversation with a large latest tool result -> Identify the live zone (newest user message + latest tool output) -> Compress the live zone type-aware, cache original in CCR (hash=def456) -> Insert marker: "compressed, retrieve: def456" -> Older turns, tools, and system prompt are forwarded byte-for-byte [Protection rules](https://docs.headroomlabs.ai/docs/context-management#protection-rules) ------------------------------------------------------------------------------------------ Headroom enforces several protections to ensure model output quality: ### [Output buffer reservation](https://docs.headroomlabs.ai/docs/context-management#output-buffer-reservation) A configurable number of tokens is reserved for the model's response. The context budget is calculated as: context_budget = model_context_limit - output_buffer_tokens This prevents the input from consuming the entire context window and leaving no room for the model to respond. ### [System message protection](https://docs.headroomlabs.ai/docs/context-management#system-message-protection) System messages are never dropped. They contain critical instructions, persona definitions, and tool descriptions that the model needs throughout the conversation. ### [Turn protection](https://docs.headroomlabs.ai/docs/context-management#turn-protection) The last N messages are always preserved, ensuring the model has immediate conversational context. The one-function `compress()` API exposes this as `CompressConfig.protect_recent`, which defaults to `4` messages (roughly the last 2 user/assistant turns) -- see `headroom/compress.py:110-112`. [Configuration](https://docs.headroomlabs.ai/docs/context-management#configuration) ------------------------------------------------------------------------------------ Context management is now automatic. Use per-request overrides to control behavior: TypeScriptPython import { compress } from "headroom-ai"; const result = await compress(messages, { model: "gpt-4o", tokenBudget: 32000, }); console.log(`Compressed: ${result.tokensBefore} -> ${result.tokensAfter}`); from headroom import HeadroomClient, OpenAIProvider from openai import OpenAI client = HeadroomClient( original_client=OpenAI(), provider=OpenAIProvider(), default_mode="optimize", ) # Per-request overrides response = client.chat.completions.create( model="gpt-4o", messages=messages, headroom_output_buffer_tokens=8000, # More room for long responses ) `HeadroomClient.chat.completions.create()` also accepts a `headroom_keep_turns` parameter, but as of this package version it is accepted and threaded through the wrapper methods without being forwarded into the transform pipeline (see `headroom/client.py` -- `TransformPipeline.apply()`'s accepted kwargs, listed in its docstring, don't include a turns/keep-turns option). Passing it does not raise an error, but it currently has no effect; use `compress()`'s `protect_recent` (see [Turn protection](https://docs.headroomlabs.ai/docs/context-management#turn-protection) above) if you need this control today. > **Note:** The `IntelligentContextConfig`, `ScoringWeights`, and `RollingWindowConfig` classes are no longer part of Headroom. Context management is now handled automatically inside the pipeline (live-zone-only compression). [Agent Orchestration\ \ Keep repeated agent wakes cache-friendly while using CCR for lossless memory digest retrieval.](https://docs.headroomlabs.ai/docs/agent-orchestration) [Persistent Memory\ \ Hierarchical, temporal memory for LLM applications. Enable your AI to remember across conversations with intelligent scoping and versioning.](https://docs.headroomlabs.ai/docs/memory) ### On this page [How It Works](https://docs.headroomlabs.ai/docs/context-management#how-it-works) [Protection rules](https://docs.headroomlabs.ai/docs/context-management#protection-rules) [Output buffer reservation](https://docs.headroomlabs.ai/docs/context-management#output-buffer-reservation) [System message protection](https://docs.headroomlabs.ai/docs/context-management#system-message-protection) [Turn protection](https://docs.headroomlabs.ai/docs/context-management#turn-protection) [Configuration](https://docs.headroomlabs.ai/docs/context-management#configuration) --- # CrewAI | Headroom CrewAI ====== Automatic tool output compression for CrewAI agents with per-tool metrics tracking. Copy MarkdownOpen Headroom integrates with [CrewAI](https://github.com/crewAIInc/crewAI) to compress tool outputs before they enter the agent's LLM context. Tool-heavy agents that return large JSON arrays, database results, or verbose logs see the largest token reductions. [Installation](https://docs.headroomlabs.ai/docs/crewai#installation) ---------------------------------------------------------------------- pip install headroom-ai crewai [Quick start](https://docs.headroomlabs.ai/docs/crewai#quick-start) -------------------------------------------------------------------- Wrap tools in one line: from crewai import Agent, Crew, Task from crewai.tools.base_tool import tool from headroom.integrations.crewai import wrap_tools_with_headroom @tool def search_database(query: str) -> str: """Search the database and return results.""" return json.dumps({"results": [...], "total": 1000}) wrapped = wrap_tools_with_headroom([search_database]) agent = Agent( role="Researcher", goal="Answer questions using data", backstory="You research things.", tools=wrapped, ) task = Task(description="Find all active users", agent=agent, expected_output="Summary") crew = Crew(agents=[agent], tasks=[task]) crew.kickoff() [Per-tool metrics](https://docs.headroomlabs.ai/docs/crewai#per-tool-metrics) ------------------------------------------------------------------------------ Track compression stats across all tool invocations: from headroom.integrations.crewai import get_tool_metrics metrics = get_tool_metrics() print(metrics.get_summary()) # { # 'total_invocations': 25, # 'total_compressions': 18, # 'total_chars_saved': 450000, # 'average_compression_ratio': 0.35, # 'by_tool': { # 'search_database': {'invocations': 15, 'compressions': 12, 'chars_saved': 320000}, # 'fetch_logs': {'invocations': 10, 'compressions': 6, 'chars_saved': 130000}, # } # } Reset between sessions: from headroom.integrations.crewai import reset_tool_metrics reset_tool_metrics() [Custom configuration](https://docs.headroomlabs.ai/docs/crewai#custom-configuration) -------------------------------------------------------------------------------------- Control the compression threshold: wrapped = wrap_tools_with_headroom( [search_database, fetch_logs], min_chars_to_compress=500, # Default: 1000 ) Use a dedicated metrics collector instead of the global one: from headroom.integrations.crewai import ToolMetricsCollector, wrap_tools_with_headroom collector = ToolMetricsCollector() wrapped = wrap_tools_with_headroom( [search_database], metrics_collector=collector, ) # After crew run print(collector.get_summary()) [Wrapping individual tools](https://docs.headroomlabs.ai/docs/crewai#wrapping-individual-tools) ------------------------------------------------------------------------------------------------ For finer control, wrap tools individually: from headroom.integrations.crewai import HeadroomToolWrapper wrapper = HeadroomToolWrapper( search_database, min_chars_to_compress=500, ) # Use wrapper directly — it's a BaseTool agent = Agent(role="Researcher", tools=[wrapper], ...) [How it works](https://docs.headroomlabs.ai/docs/crewai#how-it-works) ---------------------------------------------------------------------- CrewAI tools extend `BaseTool` with a `run()` → `_run()` execution flow. `HeadroomToolWrapper` subclasses `BaseTool` and overrides `_run()` to: 1. Call the original tool's `run()` method 2. Check if the output exceeds `min_chars_to_compress` 3. If so, compress via Headroom's `compress_tool_result()` 4. Record metrics and return the compressed output The wrapper preserves the original tool's name, description, and argument schema, so it works as a drop-in replacement anywhere CrewAI expects a tool. [Strands\ \ Context compression for Strands Agents via model wrapping and hook-based tool output compression.](https://docs.headroomlabs.ai/docs/strands) [AutoGen\ \ Automatic tool output compression for AutoGen agents with per-tool metrics tracking.](https://docs.headroomlabs.ai/docs/autogen) ### On this page [Installation](https://docs.headroomlabs.ai/docs/crewai#installation) [Quick start](https://docs.headroomlabs.ai/docs/crewai#quick-start) [Per-tool metrics](https://docs.headroomlabs.ai/docs/crewai#per-tool-metrics) [Custom configuration](https://docs.headroomlabs.ai/docs/crewai#custom-configuration) [Wrapping individual tools](https://docs.headroomlabs.ai/docs/crewai#wrapping-individual-tools) [How it works](https://docs.headroomlabs.ai/docs/crewai#how-it-works) --- # Docker-Native Install | Headroom Docker-Native Install ===================== Run Headroom without installing Python or Node.js on the host. A native \`headroom\` wrapper keeps the proxy and CLI in Docker while your other tools run on the host OS. Copy MarkdownOpen Run Headroom without installing Python or Node.js on the host. The install scripts add a native `headroom` wrapper that keeps **Headroom itself** in Docker while orchestrating the rest of your workflow on the host OS. [One-line install](https://docs.headroomlabs.ai/docs/docker-install#one-line-install) -------------------------------------------------------------------------------------- ### [Linux](https://docs.headroomlabs.ai/docs/docker-install#linux) curl -fsSL https://raw.githubusercontent.com/headroomlabs-ai/headroom/main/scripts/install.sh | bash ### [macOS (bash 4.3+)](https://docs.headroomlabs.ai/docs/docker-install#macos-bash-43) curl -fsSL https://raw.githubusercontent.com/headroomlabs-ai/headroom/main/scripts/install.sh | "$(brew --prefix bash)/bin/bash" Stock `/bin/bash` on macOS is 3.2, so install a newer bash first (for example via Homebrew) and run the installer with that shell. The installed wrapper pins that same bash interpreter so later invocations stay on the supported runtime. ### [Windows PowerShell](https://docs.headroomlabs.ai/docs/docker-install#windows-powershell) irm https://raw.githubusercontent.com/headroomlabs-ai/headroom/main/scripts/install.ps1 | iex [What the installer does](https://docs.headroomlabs.ai/docs/docker-install#what-the-installer-does) ---------------------------------------------------------------------------------------------------- 1. Verifies Docker is installed and available. 2. Pulls `ghcr.io/headroomlabs-ai/headroom:latest` by default, or reuses / pulls `HEADROOM_DOCKER_IMAGE` when you set a custom image override. 3. Installs a `headroom` wrapper into `~/.local/bin` or `~/bin`. 4. Updates shell startup files so the wrapper directory is on `PATH`. The wrapper keeps Headroom inside Docker and mounts host state back into the container so native behavior stays consistent: * project workspace → `/workspace` * `~/.headroom` * `~/.claude` * `~/.codex` * `~/.gemini` Port `8787` stays the default, so `http://localhost:8787` works the same way as a native install. Published releases also push versioned GHCR tags such as `ghcr.io/headroomlabs-ai/headroom:0.37.0`, and those images are built with the same synced package version used for the matching PyPI and npm release. [How the wrapper behaves](https://docs.headroomlabs.ai/docs/docker-install#how-the-wrapper-behaves) ---------------------------------------------------------------------------------------------------- ### [Native Headroom commands](https://docs.headroomlabs.ai/docs/docker-install#native-headroom-commands) These run directly inside the container: headroom proxy headroom learn headroom mcp install headroom memory list For `proxy`, the wrapper publishes the selected port back to the host: docker run --rm -it \ -p 8787:8787 \ -v "$PWD:/workspace" \ -w /workspace \ ghcr.io/headroomlabs-ai/headroom:latest \ headroom proxy --host 0.0.0.0 --port 8787 ### [`wrap` commands](https://docs.headroomlabs.ai/docs/docker-install#wrap-commands) `wrap` is host-oriented in Docker-native mode: * the wrapper starts the Headroom proxy in Docker * container-side prep writes Headroom config and memory into mounted host files * the target CLI itself is launched on the host by the wrapper Supported host wrap flows: * `headroom wrap claude` * `headroom wrap codex` * `headroom wrap aider` * `headroom wrap cursor` * `headroom wrap openclaw` * `headroom unwrap openclaw` OpenClaw remains host-native in Docker-native mode: * the host must already have the `openclaw` CLI installed * `headroom wrap openclaw` installs/configures the Headroom plugin through the host `openclaw` CLI * plugin auto-start still launches the installed host `headroom` wrapper from `PATH`, which then runs Headroom in Docker * local plugin source mode (`--plugin-path`) is also supported, but it may require host `npm` when build steps are needed [Persistent Docker lifecycle from the native wrapper](https://docs.headroomlabs.ai/docs/docker-install#persistent-docker-lifecycle-from-the-native-wrapper) ------------------------------------------------------------------------------------------------------------------------------------------------------------ The Docker-native `headroom` wrapper exposes the persistent Docker lifecycle directly: headroom install apply --profile default --preset persistent-docker headroom install status headroom install restart headroom install remove In Docker-native mode this surface is intentionally scoped to **persistent-docker**: * supported: `apply`, `status`, `start`, `stop`, `restart`, `remove` * supported flags: `--profile`, `--port`, `--backend`, `--anyllm-provider`, `--region`, `--mode`, `--memory`, `--no-telemetry`, `--image` * not supported: `persistent-service`, `persistent-task`, or provider/user/system mutation flags such as `--scope`, `--providers`, and `--target` Those broader lifecycle and config-mutation flows still belong to the Python-native `headroom install ...` command. Persistent Docker deployments launched by the wrapper also tag the proxy process with deployment metadata, so `/health` reports the active `profile`, `preset`, `runtime`, `supervisor`, and `scope` the same way the Python install subsystem does. [Docker Compose support](https://docs.headroomlabs.ai/docs/docker-install#docker-compose-support) -------------------------------------------------------------------------------------------------- Use `docker/docker-compose.native.yml` when you want an explicit compose-managed proxy or CLI shell, or when you prefer compose over the native wrapper's `headroom install ...` surface. ### [Persistent Docker runtime](https://docs.headroomlabs.ai/docs/docker-install#persistent-docker-runtime) The `proxy` service uses `restart: unless-stopped`, so compose can act as the always-on Docker runtime for Headroom: export HEADROOM_HOST_HOME="$HOME" export HEADROOM_WORKSPACE="$PWD" docker compose -f docker/docker-compose.native.yml up -d proxy $env:HEADROOM_HOST_HOME = $HOME $env:HEADROOM_WORKSPACE = (Get-Location).Path docker compose -f docker/docker-compose.native.yml up -d proxy This is a supported persistent-Docker path when you want the proxy managed explicitly through Compose instead of the installed wrapper. HEADROOM\_WORKSPACE vs HEADROOM\_WORKSPACE\_DIR These are two different variables — both are set by the compose file, and both are retained for backward compatibility: * **`HEADROOM_WORKSPACE`** (host-side) is the directory the compose file bind-mounts into the container as `/workspace`. It behaves like CWD in a native (non-Docker) run. * **`HEADROOM_WORKSPACE_DIR`** (inside the container) is the canonical Headroom state root from the [filesystem contract](https://docs.headroomlabs.ai/docs/filesystem-contract) . The compose file sets it to `/tmp/headroom-home/.headroom` so the proxy resolves savings, logs, TOIN, and memory under the bind-mounted `${HOME}/.headroom`. You do not need to set `HEADROOM_WORKSPACE_DIR` manually when using the shipped compose file — it is already in the `environment:` block. ### [macOS / Linux](https://docs.headroomlabs.ai/docs/docker-install#macos--linux) export HEADROOM_HOST_HOME="$HOME" export HEADROOM_WORKSPACE="$PWD" docker compose -f docker/docker-compose.native.yml up proxy ### [Windows PowerShell](https://docs.headroomlabs.ai/docs/docker-install#windows-powershell-1) $env:HEADROOM_HOST_HOME = $HOME $env:HEADROOM_WORKSPACE = (Get-Location).Path docker compose -f docker/docker-compose.native.yml up proxy You can also run one-off CLI commands through compose: docker compose -f docker/docker-compose.native.yml run --rm cli learn docker compose -f docker/docker-compose.native.yml run --rm cli mcp install [Environment passthrough](https://docs.headroomlabs.ai/docs/docker-install#environment-passthrough) ---------------------------------------------------------------------------------------------------- The wrapper forwards Headroom and provider environment variables into the container, including common prefixes such as: * `HEADROOM_` * `ANTHROPIC_` * `OPENAI_` * `GEMINI_` * `AWS_` * `GOOGLE_` / `GOOGLE_CLOUD_` * `AZURE_` * `OTEL_` That keeps provider auth and runtime config working without maintaining a separate env file for the container. [Notes](https://docs.headroomlabs.ai/docs/docker-install#notes) ---------------------------------------------------------------- * Docker is the only required Headroom runtime dependency on the host. * Wrapped tools like Claude Code, Codex CLI, Aider, and Cursor still run on the host when you use `headroom wrap ...`. * The install scripts are idempotent: rerunning them refreshes the wrapper and image without duplicating shell profile blocks. * For persistent service and task installs, use the Python-native `headroom install ...` workflow — see [Persistent Installs](https://docs.headroomlabs.ai/docs/persistent-installs) . * For Docker-native `headroom install ...`, the wrapper persists its profile manifest under `~/.headroom/deploy//`. [Next steps](https://docs.headroomlabs.ai/docs/docker-install#next-steps) -------------------------------------------------------------------------- [### Quickstart](https://docs.headroomlabs.ai/docs/quickstart) [### Proxy Server](https://docs.headroomlabs.ai/docs/proxy) [### Installation (pip / npm / plain Docker)](https://docs.headroomlabs.ai/docs/installation) [Installation\ \ Install Headroom via pip, npm, or Docker. Includes all Python extras, TypeScript setup, Docker image tags, and environment variables.](https://docs.headroomlabs.ai/docs/installation) [Persistent Installs\ \ Install Headroom as a durable local runtime — background service, scheduled watchdog, or restartable Docker container — instead of starting it ad hoc.](https://docs.headroomlabs.ai/docs/persistent-installs) ### On this page [One-line install](https://docs.headroomlabs.ai/docs/docker-install#one-line-install) [Linux](https://docs.headroomlabs.ai/docs/docker-install#linux) [macOS (bash 4.3+)](https://docs.headroomlabs.ai/docs/docker-install#macos-bash-43) [Windows PowerShell](https://docs.headroomlabs.ai/docs/docker-install#windows-powershell) [What the installer does](https://docs.headroomlabs.ai/docs/docker-install#what-the-installer-does) [How the wrapper behaves](https://docs.headroomlabs.ai/docs/docker-install#how-the-wrapper-behaves) [Native Headroom commands](https://docs.headroomlabs.ai/docs/docker-install#native-headroom-commands) [`wrap` commands](https://docs.headroomlabs.ai/docs/docker-install#wrap-commands) [Persistent Docker lifecycle from the native wrapper](https://docs.headroomlabs.ai/docs/docker-install#persistent-docker-lifecycle-from-the-native-wrapper) [Docker Compose support](https://docs.headroomlabs.ai/docs/docker-install#docker-compose-support) [Persistent Docker runtime](https://docs.headroomlabs.ai/docs/docker-install#persistent-docker-runtime) [macOS / Linux](https://docs.headroomlabs.ai/docs/docker-install#macos--linux) [Windows PowerShell](https://docs.headroomlabs.ai/docs/docker-install#windows-powershell-1) [Environment passthrough](https://docs.headroomlabs.ai/docs/docker-install#environment-passthrough) [Notes](https://docs.headroomlabs.ai/docs/docker-install#notes) [Next steps](https://docs.headroomlabs.ai/docs/docker-install#next-steps) --- # Failure Learning | Headroom Failure Learning ================ Offline failure analysis for coding agents. Analyzes past sessions, finds what went wrong, correlates with what fixed it, and writes project-level learnings. Copy MarkdownOpen `headroom learn` analyzes past coding agent sessions, finds what went wrong, correlates each failure with what eventually worked, and writes specific project-level learnings that prevent the same mistakes next session. [Quick Start](https://docs.headroomlabs.ai/docs/failure-learning#quick-start) ------------------------------------------------------------------------------ # See recommendations for current project (dry-run, no changes) headroom learn # Write recommendations to CLAUDE.local.md and MEMORY.md headroom learn --apply # Analyze a specific project headroom learn --project ~/my-project --apply # Analyze all projects headroom learn --all --apply # Write to the team-shared CLAUDE.md instead of CLAUDE.local.md headroom learn --apply --target CLAUDE.md [Success Correlation](https://docs.headroomlabs.ai/docs/failure-learning#success-correlation) ---------------------------------------------------------------------------------------------- The core innovation. Instead of cataloging failures ("Read failed 5 times"), Headroom finds what the model did to **fix** each failure: * **Failed**: `Read axion-formats/src/main/java/.../FirstClassEntity.java` * **Then succeeded**: `Read axion-scala-common/src/main/scala/.../FirstClassEntity.scala` * **Learning**: "`FirstClassEntity` is at `axion-scala-common/`, not `axion-formats/`" This produces specific, actionable corrections -- not generic advice. [What It Learns](https://docs.headroomlabs.ai/docs/failure-learning#what-it-learns) ------------------------------------------------------------------------------------ ### [Environment Facts](https://docs.headroomlabs.ai/docs/failure-learning#environment-facts) Which runtime commands work vs fail. ### Environment - **Python**: use `uv run python` (not `python3` -- modules not available outside venv) ### [File Path Corrections](https://docs.headroomlabs.ai/docs/failure-learning#file-path-corrections) Wrong paths the model keeps guessing, with the correct locations. ### File Path Corrections - `axion-common/src/.../AxionSparkConstants.scala` -> actually at `axion-spark-common/src/.../AxionSparkConstants.scala` ### [Search Scope](https://docs.headroomlabs.ai/docs/failure-learning#search-scope) Which directories to search in (narrow paths fail, broader ones work). ### Search Scope - Don't search `axion-model/` -> use `axion/` (the repo root) ### [Command Patterns](https://docs.headroomlabs.ai/docs/failure-learning#command-patterns) How commands should (and should not) be run. ### Command Patterns - **user_prefers_manual**: User rejected gradle 18 times -- show the command, don't execute - **python_runtime**: Use `uv run python` not `python3` (ModuleNotFoundError) ### [Known Large Files](https://docs.headroomlabs.ai/docs/failure-learning#known-large-files) Files that need `offset`/`limit` with Read. ### Known Large Files - `proxy/server.py` (~8000 lines) -- always use offset/limit [Where Learnings Go](https://docs.headroomlabs.ai/docs/failure-learning#where-learnings-go) -------------------------------------------------------------------------------------------- | Pattern | Destination | Why | | --- | --- | --- | | Environment, paths, search scope, commands, large files | **CLAUDE.local.md** | Personal facts (machine-specific paths), gitignored by default | | Missing paths, retry patterns, permissions | **MEMORY.md** | May change, agent-specific | `CLAUDE.local.md` lives in your project directory. It is the **personal** local memory file in Claude Code's [memory convention](https://docs.claude.com/en/docs/claude-code/memory) — meant to be gitignored — so machine-specific learnings (absolute paths, tool-discovery byproducts) don't pollute the team-shared `CLAUDE.md`. Make sure `CLAUDE.local.md` is listed in your `.gitignore`. Pass `--target CLAUDE.md` to opt into the shared file instead, or `--target ` for any custom location. MEMORY.md lives in `~/.claude/projects/*/memory/`. If an older Headroom version already wrote a learned-patterns block into your team-shared `CLAUDE.md`, the next `headroom learn --apply` moves it into `CLAUDE.local.md` and prints a warning so you can review the diff before committing. If `CLAUDE.md` contained nothing but the Headroom block, it is removed entirely. [Marker-Based Updates](https://docs.headroomlabs.ai/docs/failure-learning#marker-based-updates) ------------------------------------------------------------------------------------------------ Headroom manages a clearly-delimited section in each file: ## Headroom Learned Patterns *Auto-generated by `headroom learn` -- do not edit manually* ... On re-run, only the content between markers is replaced. Your existing file content is preserved. [Architecture](https://docs.headroomlabs.ai/docs/failure-learning#architecture) -------------------------------------------------------------------------------- The system is built with an adapter pattern so it can support multiple agent systems: * **Scanners** read tool-specific log formats (e.g., `~/.claude/projects/*.jsonl`) and produce normalized `ToolCall` sequences * **Analyzers** work on `ToolCall` data -- same analysis logic for any agent system * **Writers** output to tool-specific context injection mechanisms (e.g., CLAUDE.md) To add support for a new agent (e.g., Cursor), you write a Scanner that reads its log format and a Writer that outputs to `.cursorrules`. The analyzers stay the same. [CLI Reference](https://docs.headroomlabs.ai/docs/failure-learning#cli-reference) ---------------------------------------------------------------------------------- headroom learn [OPTIONS] Options: --project PATH Project directory to analyze (default: current directory) --all Analyze all discovered projects --apply Write recommendations (default: dry-run) --target TEXT Context file to write to, Claude Code only (default: CLAUDE.local.md). Relative to project root, or absolute. --agent TEXT Agent to analyze (e.g. claude, codex, gemini, opencode) --model TEXT LLM model to use for analysis --workers INT Number of parallel workers [Real-World Results](https://docs.headroomlabs.ai/docs/failure-learning#real-world-results) -------------------------------------------------------------------------------------------- Figures removed A prior version of this page cited specific counts (tool calls analyzed, failure rate, corrections extracted) for a real run. We could not trace those numbers to a committed benchmark, test, or dataset in this repo, and they appear to have been copied from `wiki/learn.md` rather than measured independently -- so we removed them rather than publish an unverifiable figure. Run `headroom learn` (dry-run by default) on your own session history to see real counts for your project. [SharedContext\ \ Compressed inter-agent context sharing. Reduce token usage by ~80% when agents hand off to each other.](https://docs.headroomlabs.ai/docs/shared-context) [Proxy Server\ \ Run the Headroom proxy to compress LLM traffic for any client — Claude Code, Cursor, OpenAI SDK, or custom apps.](https://docs.headroomlabs.ai/docs/proxy) ### On this page [Quick Start](https://docs.headroomlabs.ai/docs/failure-learning#quick-start) [Success Correlation](https://docs.headroomlabs.ai/docs/failure-learning#success-correlation) [What It Learns](https://docs.headroomlabs.ai/docs/failure-learning#what-it-learns) [Environment Facts](https://docs.headroomlabs.ai/docs/failure-learning#environment-facts) [File Path Corrections](https://docs.headroomlabs.ai/docs/failure-learning#file-path-corrections) [Search Scope](https://docs.headroomlabs.ai/docs/failure-learning#search-scope) [Command Patterns](https://docs.headroomlabs.ai/docs/failure-learning#command-patterns) [Known Large Files](https://docs.headroomlabs.ai/docs/failure-learning#known-large-files) [Where Learnings Go](https://docs.headroomlabs.ai/docs/failure-learning#where-learnings-go) [Marker-Based Updates](https://docs.headroomlabs.ai/docs/failure-learning#marker-based-updates) [Architecture](https://docs.headroomlabs.ai/docs/failure-learning#architecture) [CLI Reference](https://docs.headroomlabs.ai/docs/failure-learning#cli-reference) [Real-World Results](https://docs.headroomlabs.ai/docs/failure-learning#real-world-results) --- # Error Handling | Headroom Error Handling ============== How to catch and handle Headroom errors in Python and TypeScript. Error hierarchy, proxy error mapping, and safety guarantees. Copy MarkdownOpen Headroom provides explicit exceptions for debugging, with a core safety guarantee: **compression failures never break your LLM calls**. If compression fails, the original content passes through unchanged. [Error Hierarchy](https://docs.headroomlabs.ai/docs/errors#error-hierarchy) ---------------------------------------------------------------------------- TypeScriptPython HeadroomError (base class) +-- HeadroomConnectionError # Cannot reach proxy +-- HeadroomAuthError # 401 from proxy +-- HeadroomCompressError # Compression failed (with statusCode) +-- ConfigurationError # Invalid configuration +-- ProviderError # Provider issues +-- StorageError # Storage failures +-- TokenizationError # Token counting failed +-- CacheError # Cache operations failed +-- ValidationError # Validation failures +-- TransformError # Transform execution failed import { HeadroomError, HeadroomConnectionError, HeadroomAuthError, HeadroomCompressError, ConfigurationError, ProviderError, mapProxyError, } from 'headroom-ai'; HeadroomError (base class) +-- ConfigurationError # Invalid configuration +-- ProviderError # Provider issues (unknown model, etc.) +-- StorageError # Database/storage failures +-- CompressionError # Compression failures (rare) +-- TokenizationError # Token counting failed +-- CacheError # Cache operations failed +-- ValidationError # Setup validation failures +-- TransformError # Transform execution failed from headroom import ( HeadroomError, ConfigurationError, ProviderError, StorageError, CompressionError, TokenizationError, CacheError, ValidationError, TransformError, ) [Catching Errors](https://docs.headroomlabs.ai/docs/errors#catching-errors) ---------------------------------------------------------------------------- TypeScriptPython import { compress, HeadroomConnectionError, HeadroomAuthError, HeadroomCompressError, HeadroomError } from 'headroom-ai'; try { const result = await compress(messages, { model: 'gpt-4o' }); } catch (e) { if (e instanceof HeadroomConnectionError) { console.error('Cannot reach proxy:', e.message); } else if (e instanceof HeadroomAuthError) { console.error('Auth failed:', e.message); } else if (e instanceof HeadroomCompressError) { console.error(`Compress failed (${e.statusCode}):`, e.message); } else if (e instanceof HeadroomError) { console.error('Headroom error:', e.message, e.details); } } from headroom import ( HeadroomClient, HeadroomError, ConfigurationError, StorageError, ) try: client = HeadroomClient(...) response = client.chat.completions.create(...) except ConfigurationError as e: print(f"Config issue: {e}") print(f"Details: {e.details}") except StorageError as e: print(f"Storage issue: {e}") # Headroom continues to work, just without metrics persistence except HeadroomError as e: print(f"Headroom error: {e}") [Error Types in Detail](https://docs.headroomlabs.ai/docs/errors#error-types-in-detail) ---------------------------------------------------------------------------------------- ### [ConfigurationError](https://docs.headroomlabs.ai/docs/errors#configurationerror) Raised when configuration is invalid. In the Python SDK (0.37.0), `ConfigurationError` is importable and exported from `headroom`, but `HeadroomClient` itself does not currently raise it: an invalid `default_mode` string is validated by the `HeadroomMode` enum and raises a plain **`ValueError`**, not `ConfigurationError`. TypeScriptPython import { ConfigurationError } from 'headroom-ai'; // ConfigurationError is thrown when the proxy returns // a configuration_error type in its error response try: client = HeadroomClient( original_client=OpenAI(), provider=OpenAIProvider(), default_mode="invalid_mode", ) except ValueError as e: print(f"Invalid mode: {e}") # 'invalid_mode' is not a valid HeadroomMode ### [ProviderError](https://docs.headroomlabs.ai/docs/errors#providererror) Reserved for provider-specific issues, but not currently raised anywhere in the SDK. In particular, an unknown model name does **not** raise `ProviderError`: `Provider.get_context_limit()` is documented to "never raise an exception -- uses sensible defaults for unknown models," and only logs a one-time `WARNING` naming the fallback it used. import logging logging.basicConfig(level=logging.WARNING) # Does NOT raise ProviderError -- falls back to a default context limit # and logs: WARNING:headroom.providers.openai:Unknown OpenAI model # 'unknown-model-xyz': using default limit (...). To configure explicitly, # set HEADROOM_MODEL_LIMITS env var or add to ~/.headroom/models.json response = client.chat.completions.create( model="unknown-model-xyz", messages=[...], ) ### [StorageError](https://docs.headroomlabs.ai/docs/errors#storageerror) Reserved for database/storage failures. `HeadroomClient.get_metrics()` and `get_summary()` currently delegate straight to the storage backend (`headroom/client.py`) without wrapping failures in `StorageError`, so a broken database today surfaces as the underlying driver's own exception (for example `sqlite3.OperationalError`) rather than `StorageError`. Catch the base `HeadroomError` -- or `Exception` around storage calls specifically -- until that wrapping lands. try: metrics = client.get_metrics() except Exception as e: metrics = [] # Continue without historical metrics ### [CompressionError](https://docs.headroomlabs.ai/docs/errors#compressionerror) Reserved for compression failures. As of 0.37.0 this class is exported but not raised anywhere in the codebase -- there is no "strict mode" that triggers it. Compression failures are instead caught internally by broad `except Exception` handlers throughout `headroom/transforms/`, which fail open and return the original, uncompressed content. This is what backs the safety guarantee below. ### [HeadroomConnectionError (TypeScript)](https://docs.headroomlabs.ai/docs/errors#headroomconnectionerror-typescript) Raised when the TypeScript SDK cannot connect to the Headroom proxy. import { compress, HeadroomConnectionError } from 'headroom-ai'; try { await compress(messages, { model: 'gpt-4o' }); } catch (e) { if (e instanceof HeadroomConnectionError) { console.error('Is the proxy running? Start with: headroom proxy'); } } [Proxy Error Mapping](https://docs.headroomlabs.ai/docs/errors#proxy-error-mapping) ------------------------------------------------------------------------------------ The TypeScript SDK automatically maps proxy error responses to the correct error class: | HTTP Status | Proxy Error Type | TypeScript Class | | --- | --- | --- | | 401 | \-- | `HeadroomAuthError` | | 4xx/5xx | `configuration_error` | `ConfigurationError` | | 4xx/5xx | `provider_error` | `ProviderError` | | 4xx/5xx | `storage_error` | `StorageError` | | 4xx/5xx | `tokenization_error` | `TokenizationError` | | 4xx/5xx | `cache_error` | `CacheError` | | 4xx/5xx | `validation_error` | `ValidationError` | | 4xx/5xx | `transform_error` | `TransformError` | | 4xx/5xx | (other) | `HeadroomCompressError` | The `mapProxyError()` function handles this mapping: import { mapProxyError } from 'headroom-ai'; const error = mapProxyError(400, 'configuration_error', 'Invalid mode'); // Returns a ConfigurationError instance [Error Details](https://docs.headroomlabs.ai/docs/errors#error-details) ------------------------------------------------------------------------ All Headroom exceptions include a `details` dict/object with additional context: TypeScriptPython import { HeadroomError } from 'headroom-ai'; // HeadroomError.details is Record | undefined // HeadroomCompressError also has .statusCode and .errorType try: client = HeadroomClient(...) except HeadroomError as e: print(f"Error: {e}") print(f"Type: {type(e).__name__}") print(f"Details: {e.details}") # Details might include: # - field: which config field caused the error # - provider: which provider was involved # - model: which model was requested # - original_error: underlying exception [Safety Guarantee](https://docs.headroomlabs.ai/docs/errors#safety-guarantee) ------------------------------------------------------------------------------ If compression fails, the original content passes through unchanged. Your LLM calls never fail due to Headroom: messages = [\ {"role": "tool", "content": "malformed json {{{"}\ ] # This will NOT raise an exception # The malformed content passes through unchanged response = client.chat.completions.create( model="gpt-4o", messages=messages, ) [Best Practices](https://docs.headroomlabs.ai/docs/errors#best-practices) -------------------------------------------------------------------------- 1. **Catch specific exceptions** rather than broad `Exception` to avoid hiding real bugs 2. **Don't rely on storage failures raising** -- as of 0.37.0 they surface as the underlying driver's own exception, not `StorageError` (see above) 3. **Validate on startup** with `client.validate_setup()` to catch configuration issues early 4. **Enable logging** at WARNING level to see when compression falls back safely import logging logging.basicConfig(level=logging.WARNING) # WARNING:headroom.transforms.smart_crusher:SmartCrusher audit_safe: 1 protected # row(s) could not be preserved through compression (...). Failing closed: # returning original uncompressed. [Limitations\ \ When Headroom helps, when it does not, and what to watch out for. Honest documentation of compression constraints and safety gates.](https://docs.headroomlabs.ai/docs/limitations) [Troubleshooting\ \ Solutions for common Headroom issues including proxy startup, connection errors, no token savings, high latency, and installation problems.](https://docs.headroomlabs.ai/docs/troubleshooting) ### On this page [Error Hierarchy](https://docs.headroomlabs.ai/docs/errors#error-hierarchy) [Catching Errors](https://docs.headroomlabs.ai/docs/errors#catching-errors) [Error Types in Detail](https://docs.headroomlabs.ai/docs/errors#error-types-in-detail) [ConfigurationError](https://docs.headroomlabs.ai/docs/errors#configurationerror) [ProviderError](https://docs.headroomlabs.ai/docs/errors#providererror) [StorageError](https://docs.headroomlabs.ai/docs/errors#storageerror) [CompressionError](https://docs.headroomlabs.ai/docs/errors#compressionerror) [HeadroomConnectionError (TypeScript)](https://docs.headroomlabs.ai/docs/errors#headroomconnectionerror-typescript) [Proxy Error Mapping](https://docs.headroomlabs.ai/docs/errors#proxy-error-mapping) [Error Details](https://docs.headroomlabs.ai/docs/errors#error-details) [Safety Guarantee](https://docs.headroomlabs.ai/docs/errors#safety-guarantee) [Best Practices](https://docs.headroomlabs.ai/docs/errors#best-practices) --- # Image Compression | Headroom Image Compression ================= ML-powered image compression that reduces vision model token usage by 40-90% while maintaining answer accuracy. Copy MarkdownOpen Vision models charge by the token, and images are expensive. A single 1024x1024 image costs ~765 tokens on OpenAI. Headroom's image compression uses a trained ML router to analyze your query and automatically select the optimal compression technique, saving 40-90% of image tokens. [How It Works](https://docs.headroomlabs.ai/docs/image-compression#how-it-works) --------------------------------------------------------------------------------- User uploads image + asks question | [Query Analysis] TrainedRouter (MiniLM from HuggingFace) Classifies: "What animal is this?" -> full_low | [Image Analysis] SigLIP analyzes image properties (has text? complex? fine details?) | [Apply Compression] OpenAI: detail="low" Anthropic: Resize to 512px Google: Resize to 768px | Compressed request to LLM The router is a fine-tuned MiniLM classifier (`chopratejas/technique-router` on HuggingFace) with 93.7% accuracy across 1,157 training examples. [Compression Techniques](https://docs.headroomlabs.ai/docs/image-compression#compression-techniques) ----------------------------------------------------------------------------------------------------- | Technique | Savings | When Used | Example Query | | --- | --- | --- | --- | | `full_low` | ~87% | General understanding | "What is this?", "Describe the scene" | | `preserve` | 0% | Fine details needed | "Count the whiskers", "Read the serial number" | | `crop` | 50-90% | Region-specific queries | "What's in the corner?", "Focus on the background" | | `transcode` | ~99% | Text extraction | "Read the sign", "Transcribe the document" | [Quick Start](https://docs.headroomlabs.ai/docs/image-compression#quick-start) ------------------------------------------------------------------------------- ### [With Headroom Proxy (Zero Code Changes)](https://docs.headroomlabs.ai/docs/image-compression#with-headroom-proxy-zero-code-changes) # Start the proxy headroom proxy --port 8787 # Connect your client -- images are compressed automatically ANTHROPIC_BASE_URL=http://localhost:8787 claude ### [With HeadroomClient](https://docs.headroomlabs.ai/docs/image-compression#with-headroomclient) from headroom import HeadroomClient, OpenAIProvider from openai import OpenAI client = HeadroomClient( original_client=OpenAI(), provider=OpenAIProvider(), default_mode="optimize", ) response = client.chat.completions.create( model="gpt-4o", messages=[{\ "role": "user",\ "content": [\ {"type": "text", "text": "What animal is this?"},\ {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}\ ]\ }] ) # Image automatically compressed with detail="low" (87% savings) ### [Direct API](https://docs.headroomlabs.ai/docs/image-compression#direct-api) from headroom.image import ImageCompressor compressor = ImageCompressor() # Compress images in messages compressed_messages = compressor.compress(messages, provider="openai") # Check savings print(f"Saved {compressor.last_savings:.0f}% tokens") print(f"Technique: {compressor.last_result.technique.value}") [Provider Support](https://docs.headroomlabs.ai/docs/image-compression#provider-support) ----------------------------------------------------------------------------------------- The compressor adapts its strategy per provider: | Provider | Compression Method | Details | | --- | --- | --- | | **OpenAI** | Sets `detail="low"` | Native detail parameter | | **Anthropic** | Resizes to 512px | PIL-based resize | | **Google Gemini** | Resizes to 768px | Optimized for Gemini's 768x768 tile system | ### [Token Savings by Provider](https://docs.headroomlabs.ai/docs/image-compression#token-savings-by-provider) **OpenAI** (1024x1024 image): | Technique | Before | After | Savings | | --- | --- | --- | --- | | `full_low` | 765 tokens | 85 tokens | 89% | | `preserve` | 765 tokens | 765 tokens | 0% | **Anthropic** (1024x1024 image): | Before | After | Savings | | --- | --- | --- | | ~1,398 tokens | ~349 tokens | 75% | **Google Gemini** (1536x1536 image): | Before | After | Savings | | --- | --- | --- | | 1,032 tokens (4 tiles) | 258 tokens (1 tile) | 75% | Unlike the OpenAI and Anthropic figures above (computed from the formulas in `headroom/image/tile_optimizer.py` and verified to match this page's numbers exactly), Headroom has no Gemini token-counting formula in the codebase -- these figures describe Google's publicly documented tile pricing, not something this repo computes or tests. [Configuration](https://docs.headroomlabs.ai/docs/image-compression#configuration) ----------------------------------------------------------------------------------- from headroom.image import ImageCompressor compressor = ImageCompressor( model_id="chopratejas/technique-router", # HuggingFace model use_siglip=True, # Enable image analysis device="cuda", # Use GPU if available (auto, cuda, cpu, mps) ) ### [Proxy Configuration](https://docs.headroomlabs.ai/docs/image-compression#proxy-configuration) # Image optimization is part of the ContentRouter path when the optional image # dependencies are installed. There are no public proxy CLI image toggles in # the current release. headroom proxy [Performance](https://docs.headroomlabs.ai/docs/image-compression#performance) ------------------------------------------------------------------------------- | Metric | Value | | --- | --- | | Image resize | ~5-20ms (measured ~9ms median for a 1024px -> 512px PIL/LANCZOS resize) | | First request | Model download, cached after (duration depends on network) | | Router accuracy | 93.7% (fine-tuned MiniLM classifier, 1,157 training examples -- see `headroom/image/trained_router.py`) | | Model size | ~100MB (PyTorch MiniLM router) or ~32MB (ONNX INT8 router, preferred in production) | | GPU memory (SigLIP) | ~400MB | Automatic with the proxy When using the Headroom proxy, image compression happens automatically on every request that contains images. No code changes needed. [Code Compression\ \ AST-aware compression that preserves imports, signatures, and types while compressing function bodies. Powered by tree-sitter.](https://docs.headroomlabs.ai/docs/code-compression) [Text & Log Compression\ \ Specialized compressors for search results, build logs, diffs, and general text. Each preserves what matters for its content type.](https://docs.headroomlabs.ai/docs/text-and-logs) ### On this page [How It Works](https://docs.headroomlabs.ai/docs/image-compression#how-it-works) [Compression Techniques](https://docs.headroomlabs.ai/docs/image-compression#compression-techniques) [Quick Start](https://docs.headroomlabs.ai/docs/image-compression#quick-start) [With Headroom Proxy (Zero Code Changes)](https://docs.headroomlabs.ai/docs/image-compression#with-headroom-proxy-zero-code-changes) [With HeadroomClient](https://docs.headroomlabs.ai/docs/image-compression#with-headroomclient) [Direct API](https://docs.headroomlabs.ai/docs/image-compression#direct-api) [Provider Support](https://docs.headroomlabs.ai/docs/image-compression#provider-support) [Token Savings by Provider](https://docs.headroomlabs.ai/docs/image-compression#token-savings-by-provider) [Configuration](https://docs.headroomlabs.ai/docs/image-compression#configuration) [Proxy Configuration](https://docs.headroomlabs.ai/docs/image-compression#proxy-configuration) [Performance](https://docs.headroomlabs.ai/docs/image-compression#performance) --- # How Compression Works | Headroom How Compression Works ===================== Understand Headroom's compression pipeline, automatic content routing, and how different content types are compressed. Copy MarkdownOpen Headroom automatically detects what kind of content you're sending and routes it to the right compressor. You don't need to configure anything -- just call `compress()` and the pipeline handles the rest. [The Pipeline](https://docs.headroomlabs.ai/docs/how-compression-works#the-pipeline) ------------------------------------------------------------------------------------- Every request flows through a short pipeline: ┌──────────────┐ ┌────────────────┐ │ CacheAligner │────>│ ContentRouter │ │ (off by │ │ │ │ default) │ │ Detect type & │ │ report drift │ │ route to best │ │ for cache │ │ compressor │ └──────────────┘ └────────────────┘ 1. **CacheAligner** _(detector-only, off by default)_ reports dynamic-prefix drift (dates, session context) so callers can keep the static prefix cacheable. It never rewrites your messages. 2. **ContentRouter** inspects each content block and routes it to one compressor -- SmartCrusher for JSON arrays, CodeAwareCompressor for source code, LogCompressor for build output, and so on. This is where essentially all compression happens. [Content Type Detection](https://docs.headroomlabs.ai/docs/how-compression-works#content-type-detection) --------------------------------------------------------------------------------------------------------- The router auto-detects content type by analyzing structure and patterns. No manual hints required. | Content Type | Detection Signal | Compressor | Typical Savings | | --- | --- | --- | --- | | JSON arrays | Valid JSON with array elements | SmartCrusher | Varies with redundancy -- measured 26-54% on realistic (non-duplicate) tool-output arrays of 100-10,000 items; highly repetitive arrays compress far more. See `benchmarks/compression_benchmark.py`. | | Source code | Syntax patterns, indentation, keywords | Kompress via the library `compress()`/`HeadroomClient` path (`enable_code_aware` defaults to `False` there); CodeAwareCompressor via `headroom proxy` (`HEADROOM_CODE_AWARE_ENABLED` defaults to on) | workload-dependent | | Search results | `file:line:content` format | SearchCompressor | 80-95% (measured ~91% on a synthetic 40-file ripgrep-style corpus) | | Build/test logs | Timestamps, log levels, pytest/npm markers | LogCompressor | 85-95% (measured ~93% on a synthetic 1,000-line log with clustered errors) | | Diffs | Unified diff format | DiffCompressor | 60-80% (measured ~70% on a multi-file diff with default `max_context_lines`/`max_hunks_per_file` limits) | | HTML | Tag structure | HTMLExtractor | 70-90% (module docstring; measured ~81% on a synthetic nav/ads/script-heavy page) | | Tabular (CSV/TSV/markdown tables) | Delimited rows / table syntax | TabularCompressor | Varies with redundancy -- TabularCompressor bridges to SmartCrusher, so it inherits the same range (measured 26-28% on non-duplicate rows) | | Structured config (YAML/TOML/INI) | Config syntax | ConfigCompressor | 40-70% (measured ~50% on a commented/blank-line-heavy YAML file) | | Plain text | Text with no stronger signal | Kompress | workload-dependent | | Anything else | Fallback chain, then passthrough if no safe saving is found | Kompress / passthrough | varies | [Quick Start](https://docs.headroomlabs.ai/docs/how-compression-works#quick-start) ----------------------------------------------------------------------------------- TypeScriptPython import { compress } from "headroom-ai"; const messages = [\ { role: "system" as const, content: "You are a helpful assistant." },\ { role: "user" as const, content: "Summarize this data" },\ { role: "tool" as const, content: '{"results": [...]}', tool_call_id: "call_1" },\ ]; const result = await compress(messages); console.log(`Tokens saved: ${result.tokensSaved}`); console.log(`Compression ratio: ${result.compressionRatio}`); from headroom import compress messages = [\ {"role": "user", "content": "Summarize the tool output."},\ {"role": "tool", "tool_call_id": "call_1", "content": large_output},\ ] result = compress(messages, model="gpt-4o") print(result.messages) print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})") [Configuring the Compressor](https://docs.headroomlabs.ai/docs/how-compression-works#configuring-the-compressor) ----------------------------------------------------------------------------------------------------------------- TypeScriptPython import { compress } from "headroom-ai"; const result = await compress(messages, { model: "gpt-4o", tokenBudget: 50000, }); console.log(`Before: ${result.tokensBefore} tokens`); console.log(`After: ${result.tokensAfter} tokens`); console.log(`Transforms: ${result.transformsApplied.join(", ")}`); from headroom import CompressConfig, compress config = CompressConfig( compress_user_messages=True, protect_recent=0, target_ratio=0.5, # Kompress keep ratio; structural compressors use their own rules min_tokens_to_compress=250, ) result = compress(messages, model="gpt-4o", config=config) print(f"Saved: {result.tokens_saved} tokens") print(result.transforms_applied) [Structure Preservation](https://docs.headroomlabs.ai/docs/how-compression-works#structure-preservation) --------------------------------------------------------------------------------------------------------- Headroom doesn't blindly truncate. It identifies what matters in each content type and preserves it: | Content Type | What's Preserved | What's Compressed | | --- | --- | --- | | **JSON** | Keys, brackets, booleans, nulls, short values, UUIDs | Long string values, whitespace | | **Code** | Imports, function signatures, class definitions, types | Function bodies, comments | | **Logs** | Timestamps, log levels, error messages, stack traces | Repeated patterns, verbose details | | **Text** | High-entropy tokens (IDs, hashes), headers | Low-information content | [Real Compression Ratios](https://docs.headroomlabs.ai/docs/how-compression-works#real-compression-ratios) ----------------------------------------------------------------------------------------------------------- Compression ratio and latency both depend heavily on the shape of the input (how redundant the data is, how large the payload is, and -- for Kompress -- what `target_ratio` is configured). There's no single number that holds across workloads, so treat the ranges in the table above as directional and reproduce them on your own traffic with the harnesses in [`benchmarks/`](https://github.com/headroomlabs-ai/headroom/tree/main/benchmarks) (for example `compression_benchmark.py`, `bench_transforms.py`, `text_crusher_quality_eval.py`). What stays constant across content types is _what's preserved_: | Content Type | What's Preserved | | --- | --- | | JSON (large arrays) | All keys, structure | | Source code (Python) | Signatures, imports | | Search results | Relevant matches | | Build logs | Errors, stack traces | | Plain text | High-entropy tokens | [Compressing multiple documents](https://docs.headroomlabs.ai/docs/how-compression-works#compressing-multiple-documents) ------------------------------------------------------------------------------------------------------------------------- The public `compress()` API accepts an LLM message list. Put documents in separate messages or call it once per independent request; this preserves the same message shape you will send to the provider: from headroom import compress messages = [\ {"role": "user", "content": "Compare these documents."},\ {"role": "tool", "tool_call_id": "doc_1", "content": first_document},\ {"role": "tool", "tool_call_id": "doc_2", "content": second_document},\ ] result = compress(messages, model="gpt-4o") [What Happens Under the Hood](https://docs.headroomlabs.ai/docs/how-compression-works#what-happens-under-the-hood) ------------------------------------------------------------------------------------------------------------------- When you call `compress()`, here is the full sequence: 1. **Safety gates** -- protected tools, recent turns, frozen cache prefixes, and minimum-size rules decide what may change. 2. **Content detection** -- Magika when available plus deterministic pattern matching identifies the content type. 3. **Structural compression** -- format-specific handlers compact JSON, logs, search results, tables, configuration, HTML, and other recognized structures. 4. **Fallback compression** -- Kompress handles eligible text when its runtime is available; a no-saving or failed transform passes the original through. 5. **CCR storage in proxy flows** -- when CCR is enabled, originals for marker-bearing compression are retained for later retrieval. `--lossless` and `--no-ccr` intentionally use different contracts; see [Reversible Compression](https://docs.headroomlabs.ai/docs/ccr) . Zero-config by default The pipeline works out of the box with no configuration. All detection, routing, and compression happens automatically. Configuration is available when you need fine-grained control. [Persistent Installs\ \ Install Headroom as a durable local runtime — background service, scheduled watchdog, or restartable Docker container — instead of starting it ad hoc.](https://docs.headroomlabs.ai/docs/persistent-installs) [SmartCrusher\ \ Statistical JSON and array compression that keeps important items and drops the rest.](https://docs.headroomlabs.ai/docs/smart-crusher) ### On this page [The Pipeline](https://docs.headroomlabs.ai/docs/how-compression-works#the-pipeline) [Content Type Detection](https://docs.headroomlabs.ai/docs/how-compression-works#content-type-detection) [Quick Start](https://docs.headroomlabs.ai/docs/how-compression-works#quick-start) [Configuring the Compressor](https://docs.headroomlabs.ai/docs/how-compression-works#configuring-the-compressor) [Structure Preservation](https://docs.headroomlabs.ai/docs/how-compression-works#structure-preservation) [Real Compression Ratios](https://docs.headroomlabs.ai/docs/how-compression-works#real-compression-ratios) [Compressing multiple documents](https://docs.headroomlabs.ai/docs/how-compression-works#compressing-multiple-documents) [What Happens Under the Hood](https://docs.headroomlabs.ai/docs/how-compression-works#what-happens-under-the-hood) --- # Filesystem Contract | Headroom Filesystem Contract =================== Where Headroom writes config, runtime state, logs, and caches — the canonical two-root model, precedence rules, and Docker behavior. Copy MarkdownOpen Headroom writes configuration, runtime state, logs, and caches to a small set of well-known paths under the user's home directory. This page is the source of truth for where those paths live, how to override them, and how they behave inside Docker containers. [Two-root model](https://docs.headroomlabs.ai/docs/filesystem-contract#two-root-model) --------------------------------------------------------------------------------------- | Variable | Default | Purpose | Typical access | | --- | --- | --- | --- | | `HEADROOM_CONFIG_DIR` | `~/.headroom/config` | User/admin-authored configuration (model catalogs, plugin settings, etc.) | Read-mostly | | `HEADROOM_WORKSPACE_DIR` | `~/.headroom` | Runtime state written by the proxy and CLI (savings, logs, memory DB, telemetry, caches) | Read-write | Both variables are recognized by the Python proxy / CLI and the npm SDK. They are **additive** — every pre-existing per-resource env var (`HEADROOM_SAVINGS_PATH`, `HEADROOM_TOIN_PATH`, `HEADROOM_SUBSCRIPTION_STATE_PATH`, `HEADROOM_MODEL_LIMITS`, ...) continues to work with identical semantics. [Precedence](https://docs.headroomlabs.ai/docs/filesystem-contract#precedence) ------------------------------------------------------------------------------- For every per-resource helper, resolution follows this order: explicit argument │ falls through when None/"" ▼ per-resource env var (e.g. HEADROOM_SAVINGS_PATH) │ falls through when unset/blank ▼ derived from canonical root │ e.g. ${HEADROOM_WORKSPACE_DIR}/proxy_savings.json ▼ default (e.g. ~/.headroom/proxy_savings.json) Examples: * `HEADROOM_WORKSPACE_DIR=/mnt/state` → savings land at `/mnt/state/proxy_savings.json` unless `HEADROOM_SAVINGS_PATH` overrides. * `HEADROOM_SAVINGS_PATH=/custom/savings.json` always wins, even when `HEADROOM_WORKSPACE_DIR` is set. * Unset both and the default is `~/.headroom/proxy_savings.json`. [Bucket assignments](https://docs.headroomlabs.ai/docs/filesystem-contract#bucket-assignments) ----------------------------------------------------------------------------------------------- ### [Workspace bucket (`HEADROOM_WORKSPACE_DIR`)](https://docs.headroomlabs.ai/docs/filesystem-contract#workspace-bucket-headroom_workspace_dir) | Resource | Default path | Legacy env var | | --- | --- | --- | | Proxy savings ledger | `${WORKSPACE_DIR}/proxy_savings.json` | `HEADROOM_SAVINGS_PATH` | | TOIN telemetry JSON | `${WORKSPACE_DIR}/toin.json` | `HEADROOM_TOIN_PATH` | | Subscription tracker state | `${WORKSPACE_DIR}/subscription_state.json` | `HEADROOM_SUBSCRIPTION_STATE_PATH` | | Memory SQLite | `${WORKSPACE_DIR}/memory.db` | CLI `--memory-db-path` | | Native memory directory | `${WORKSPACE_DIR}/memories/` | `MemoryConfig.native_memory_dir` | | License cache | `${WORKSPACE_DIR}/license_cache.json` | — | | Session stats JSONL | `${WORKSPACE_DIR}/session_stats.jsonl` | — | | Memory sync state | `${WORKSPACE_DIR}/sync_state.json` | — | | Memory bridge state | `${WORKSPACE_DIR}/bridge_state.json` | — | | Proxy log directory | `${WORKSPACE_DIR}/logs/` | — | | HTTP 400 debug dumps | `${WORKSPACE_DIR}/logs/debug_400/` | — | | Deployment profiles | `${WORKSPACE_DIR}/deploy/` | — | | Beacon lock file | `${WORKSPACE_DIR}/.beacon_lock_` | — | ### [Config bucket (`HEADROOM_CONFIG_DIR`)](https://docs.headroomlabs.ai/docs/filesystem-contract#config-bucket-headroom_config_dir) | Resource | Default path | Legacy env var | | --- | --- | --- | | Models catalog | `${CONFIG_DIR}/models.json` | `HEADROOM_MODEL_LIMITS` (content override) | | Plugin settings | `${CONFIG_DIR}/plugins//...` | — | ### [Backward compatibility — models.json](https://docs.headroomlabs.ai/docs/filesystem-contract#backward-compatibility--modelsjson) `models.json` historically lived at `~/.headroom/models.json` (i.e. in the workspace root, not in `config/`). For a seamless migration the Python providers check **both** locations in this order: 1. `${HEADROOM_CONFIG_DIR}/models.json` (new canonical location) 2. `${HEADROOM_WORKSPACE_DIR}/models.json` (legacy fallback) Existing installs continue to work unchanged. New installs are encouraged to put `models.json` in the config bucket. [Plugin authors](https://docs.headroomlabs.ai/docs/filesystem-contract#plugin-authors) --------------------------------------------------------------------------------------- Two helpers give plugins isolated, per-plugin directories under both roots: ### [Python](https://docs.headroomlabs.ai/docs/filesystem-contract#python) from headroom import paths cfg_dir = paths.plugin_config_dir("my-plugin") # → ~/.headroom/config/plugins/my-plugin state_dir = paths.plugin_workspace_dir("my-plugin") # → ~/.headroom/plugins/my-plugin cfg_dir.mkdir(parents=True, exist_ok=True) (cfg_dir / "settings.json").write_text("{}") ### [npm SDK](https://docs.headroomlabs.ai/docs/filesystem-contract#npm-sdk) import { pluginConfigDir, pluginWorkspaceDir } from "@headroom/sdk"; const cfgDir = pluginConfigDir("my-plugin"); const stateDir = pluginWorkspaceDir("my-plugin"); Plugin-author helpers reject names containing `/` or `\` to keep the namespace flat. [Docker naming overlap: `HEADROOM_WORKSPACE` vs `HEADROOM_WORKSPACE_DIR`](https://docs.headroomlabs.ai/docs/filesystem-contract#docker-naming-overlap-headroom_workspace-vs-headroom_workspace_dir) ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- These are **two different variables** with different semantics, both retained for backward compatibility: | Variable | Scope | Meaning | | --- | --- | --- | | `HEADROOM_WORKSPACE` | Host-side (Docker) | Directory to bind-mount into the container as `/workspace` (equivalent to CWD in native runs). Used by `docker-compose.native.yml`. | | `HEADROOM_WORKSPACE_DIR` | Inside the container | Canonical Headroom state root. Resolves to `/tmp/headroom-home/.headroom` inside the official container image, which in turn bind-mounts to `${HOME}/.headroom` on the host. | The official Docker bootstrap (compose file, `scripts/install.sh`, and the Python `install` command) sets `HEADROOM_WORKSPACE_DIR` and `HEADROOM_CONFIG_DIR` inside the container so the proxy resolves state to the bind-mounted path without any user action. [Project-scoped `.headroom/` directories](https://docs.headroomlabs.ai/docs/filesystem-contract#project-scoped-headroom-directories) ------------------------------------------------------------------------------------------------------------------------------------- A few code paths deliberately use **project-local** `.headroom/` paths resolved relative to the current working directory rather than the canonical workspace root: * `headroom/proxy/server.py` — project-scoped memory DB default * `headroom/memory/mcp_server.py` — project-scoped memory DB default * `headroom/cli/wrap.py` — project-scoped memory and hook artifacts These **do not obey** `HEADROOM_WORKSPACE_DIR`. This is intentional: it preserves the "project memory lives in the project directory" invariant. Users who want a single centrally located memory store can pass `--memory-db-path ` explicitly or set the path via the plugin API. [Legacy per-resource env vars](https://docs.headroomlabs.ai/docs/filesystem-contract#legacy-per-resource-env-vars) ------------------------------------------------------------------------------------------------------------------- Every legacy env var continues to work with its original semantics (raw string in, raw string out — no tilde expansion, no path-separator normalization), ensuring byte-for-byte backward compatibility. Full list: * `HEADROOM_SAVINGS_PATH` * `HEADROOM_TOIN_PATH` * `HEADROOM_SUBSCRIPTION_STATE_PATH` * `HEADROOM_MODEL_LIMITS` (content override — JSON string or file path) [See also](https://docs.headroomlabs.ai/docs/filesystem-contract#see-also) --------------------------------------------------------------------------- [### Configuration](https://docs.headroomlabs.ai/docs/configuration) [### Docker-Native Install](https://docs.headroomlabs.ai/docs/docker-install) [### Persistent Installs](https://docs.headroomlabs.ai/docs/persistent-installs) [### Memory](https://docs.headroomlabs.ai/docs/memory) [Pipeline Extensions\ \ Write a request-normalization extension for a quirky upstream provider, and route requests to different upstream bases per request with x-headroom-base-url.](https://docs.headroomlabs.ai/docs/pipeline-extensions) [Savings Tracking\ \ Durable, over-time compression savings — cost avoided plus Today / Last 7 days / Last 30 days and per-model / per-client breakdowns via \`headroom savings\`.](https://docs.headroomlabs.ai/docs/savings) ### On this page [Two-root model](https://docs.headroomlabs.ai/docs/filesystem-contract#two-root-model) [Precedence](https://docs.headroomlabs.ai/docs/filesystem-contract#precedence) [Bucket assignments](https://docs.headroomlabs.ai/docs/filesystem-contract#bucket-assignments) [Workspace bucket (`HEADROOM_WORKSPACE_DIR`)](https://docs.headroomlabs.ai/docs/filesystem-contract#workspace-bucket-headroom_workspace_dir) [Config bucket (`HEADROOM_CONFIG_DIR`)](https://docs.headroomlabs.ai/docs/filesystem-contract#config-bucket-headroom_config_dir) [Backward compatibility — models.json](https://docs.headroomlabs.ai/docs/filesystem-contract#backward-compatibility--modelsjson) [Plugin authors](https://docs.headroomlabs.ai/docs/filesystem-contract#plugin-authors) [Python](https://docs.headroomlabs.ai/docs/filesystem-contract#python) [npm SDK](https://docs.headroomlabs.ai/docs/filesystem-contract#npm-sdk) [Docker naming overlap: `HEADROOM_WORKSPACE` vs `HEADROOM_WORKSPACE_DIR`](https://docs.headroomlabs.ai/docs/filesystem-contract#docker-naming-overlap-headroom_workspace-vs-headroom_workspace_dir) [Project-scoped `.headroom/` directories](https://docs.headroomlabs.ai/docs/filesystem-contract#project-scoped-headroom-directories) [Legacy per-resource env vars](https://docs.headroomlabs.ai/docs/filesystem-contract#legacy-per-resource-env-vars) [See also](https://docs.headroomlabs.ai/docs/filesystem-contract#see-also) --- # Grok Build Integration | Headroom Grok Build Integration ====================== Route Grok Build traffic through Headroom for token compression and per-project savings attribution. Copy MarkdownOpen Use `headroom wrap grok-build` to route Grok Build LLM traffic through the local Headroom proxy. The wrapper starts or reuses the proxy, injects a reversible `[model.grok-build]` override into `~/.grok/config.toml` (or `$GROK_HOME/config.toml`), and prints next steps for launching `grok`. [Quick Start](https://docs.headroomlabs.ai/docs/grok-build#quick-start) ------------------------------------------------------------------------ headroom wrap grok-build In another terminal, from the same project directory: grok When you are done: headroom unwrap grok-build [What `wrap grok-build` Does](https://docs.headroomlabs.ai/docs/grok-build#what-wrap-grok-build-does) ------------------------------------------------------------------------------------------------------ | Step | What happens | | --- | --- | | Proxy | Starts the Headroom proxy unless `--no-proxy` is set | | Model config | Writes or updates `[model.grok-build] base_url` in Grok's `config.toml`, pointing at `http://127.0.0.1:/v1` (with optional `/p/` prefix for savings attribution) | | Existing config | If you already have a `[model.grok-build]` table, Headroom rewrites `base_url` in place instead of appending a duplicate table (invalid TOML) | | MCP install | `headroom mcp install` can register Headroom MCP via `GrokRegistrar` | | Backup | Snapshots `config.toml` to `config.toml.headroom-backup` before the first injection | [Options](https://docs.headroomlabs.ai/docs/grok-build#options) ---------------------------------------------------------------- headroom wrap grok-build \ --port 8787 \ # Proxy port (default: 8787) --no-proxy \ # Use an existing proxy instead of starting one --learn \ # Enable live traffic learning --memory # Enable persistent memory [Environment Variables](https://docs.headroomlabs.ai/docs/grok-build#environment-variables) -------------------------------------------------------------------------------------------- | Variable | Description | | --- | --- | | `GROK_HOME` | Override Grok config directory (default: `~/.grok`) | | `XAI_API_KEY` | Grok API key (also accepts `GROK_CODE_XAI_API_KEY`) | [Persistent Install](https://docs.headroomlabs.ai/docs/grok-build#persistent-install) -------------------------------------------------------------------------------------- `grok_build` is an install target for `headroom install apply --providers manual --target grok_build`. The install manifest records proxy env values for Grok Build alongside other wrapped agents. [Unwrap](https://docs.headroomlabs.ai/docs/grok-build#unwrap) -------------------------------------------------------------- `headroom unwrap grok-build` restores the pre-wrap `config.toml` from backup when available, or strips Headroom marker blocks and in-place `base_url` rewrites when no backup exists. [OpenCode + DeepSeek\ \ Configure OpenCode to route DeepSeek traffic through the Headroom proxy for compression, output shaping, and savings visibility.](https://docs.headroomlabs.ai/docs/opencode-deepseek) [MCP Tools\ \ Compression, retrieval, and stats as MCP tools for Claude Code, Cursor, and any MCP-compatible host.](https://docs.headroomlabs.ai/docs/mcp) ### On this page [Quick Start](https://docs.headroomlabs.ai/docs/grok-build#quick-start) [What `wrap grok-build` Does](https://docs.headroomlabs.ai/docs/grok-build#what-wrap-grok-build-does) [Options](https://docs.headroomlabs.ai/docs/grok-build#options) [Environment Variables](https://docs.headroomlabs.ai/docs/grok-build#environment-variables) [Persistent Install](https://docs.headroomlabs.ai/docs/grok-build#persistent-install) [Unwrap](https://docs.headroomlabs.ai/docs/grok-build#unwrap) ---