Skip to Content
StudioContext DesignContext-Pipe

Context-Pipe

Context-Pipe is the high-performance orchestration layer designed to bring the Unix Philosophy to the AI context window. It serves as the universal switchboard that connects raw data sources, shell utilities, and MCP tools to intelligence refineries.

githubluismichio/context-pipe
CI statusTestsLicense

The Vision: Systems over Patches

The AI agent has a fundamental infrastructure problem: every tool call returns raw, unfiltered output directly into the context window. context-pipe solves this at the infrastructure layer—before the LLM sees anything.

In the Studio of Two philosophy, we build Systems, not Patches. A patch would be a custom filter per tool. A system is a universal protocol: any tool that reads stdin and writes stdout becomes a node. Any sequence of nodes becomes a pipe.

Core Components

1. The Context-Pipe Protocol (CPP)

CPP is a language-agnostic standard based on stdin and stdout. If a tool can read text and emit text, it can be a node in the pipe. This allows for extreme decoupling of infrastructure:

  • Binary Nodes: Local tools like markitdown or prettier.
  • Shell Nodes: Standard Unix utilities like grep, awk, or jq.
  • MCP Nodes: Any protocol-compliant tool call (GitHub, Firecrawl, Slack).
  • Script Nodes: Project-specific Python or JS logic.

2. The Orchestration Spine

The orchestrator manages the lifecycle of a pipe run, handling process spawning, timeout guards, and signal propagation. It ensures that data flows through the chain without being prematurely leaked to the context window.

3. The Universal Switchboard (pipes.json)

A centralized registry that defines how tools map to specific refineries. It allows the agent to automatically route a git diff through a log sifter and a documentation fetch through a semantic refinery based on simple configuration.

Example: The Context Supply Chain

Data enters raw, passes through a sequence of refineries, and arrives at the LLM as dense, high-signal content.

[Source Data]
  -> firecrawl/scrape      # Fetch live page as clean text
  -> markitdown            # Convert to structured Markdown
  -> rg 'error|critical'   # Surface only relevant sections
  -> semantic-sift-cli     # Distil to high-signal summary
  -> LLM Reasoning         # Final refined signal
  • Agnostic: Chain Rust, Python, Go, and Bash tools into a single stream.
  • Deterministic: Use native OS utilities (Grep, JQ, Sed) for perfect data filtering.
  • Composable: Build complex “Mental Supply Chains” using a simple JSON configuration.

Advanced Node Chaining

Context-Pipe supports multi-node data streams that can transform context in real-time:

  1. Binary Nodes: Standalone tools like the native Rust sift-core.
  2. Bash Nodes: Direct execution of shell commands for high-speed filtering.
  3. Skill Nodes: Injection of expert “Instruction Lenses” (e.g., Security Auditor, UI Expert) into the context stream.

Resilient Orchestration: DAG Routing & Self-Healing

While simple Unix pipelines are linear, real-world context orchestration requires handling conditional state transitions and recovery loops. The orchestrator supports branch-aware topology definitions:

1. Conditional Validator Nodes

By declaring a node with type: "validator", the orchestrator evaluates its process exit code to route execution dynamically to named branches. This replaces imperative script routing with declarative topology maps in pipes.json.

2. In-Run Self-Healing

If a validation or path gate fails (e.g., a spec file is missing on a refresh cycle), the orchestrator can intercept the error and execute a predefined bootstrap recovery chain before routing back to the main pipeline.

3. Protocol Hygiene (Banner Tolerance)

To support non-conformant MCP servers that print startup headers or banners to stdout before initiating JSON-RPC communication, the orchestrator automatically filters out pre-handshake stdout noise, protecting the communication channel from JSON parsing errors.

Subconscious Interceptors

Context-Pipe acts as an invisible hook within the agentic ecosystem. It automatically intercepts tool responses in Cursor, Windsurf, Claude Code, and VS Code, sifting the data through your custom pipes before the LLM ever sees it.

The Context Balance Sheet

CPP includes a dedicated telemetry engine that tracks the ROI of your context window. Every character saved is tracked, allowing you to audit the “Net Gain” in signal across every individual node in your supply chain.