pi-hydra: Mob Programming for Coding Agents

Explore pi-hydra, an extension for coding agents that uses observer "heads" to review work in real-time. Learn how prompt caching makes this cost-effective, adding only 30% to session costs.

pi Anthropic Messages API Prompt caching TypeScript

Overview

pi-hydra is an open-source extension for the pi coding agent that adds observer “heads”. A head reviews the agent’s work while the agent is still working. Each head is one markdown file with its own focus (security, quality, keeping docs current) and sees exactly what the agent sees. After every step it decides: stay quiet, print a note for me, queue feedback for the next turn, steer the agent mid-run, or interrupt the run entirely.

The part that makes this cheap is prompt caching. Heads replay the agent’s own provider requests byte-for-byte at the moments the cache commits, so an observation is a near-pure cache read instead of a full-price context rebuild. Measured over real sessions an always-on head adds about 30% to session cost.

Live demo: an agent builds a small app while a security head catches an RCE mid-run and steers the fix into the conversation. I’ll show the head files, live cache-hit stats and the experiments harness that re-verifies the cache claims against the live API for under a dollar.

Links

https://github.com/pandysp/pi-hydra
TypeScript project managing mid-flight LLM stream interruptions without cache parity.

Tech stack

pi

Pi is a personal AI designed for supportive, empathetic, and fluid conversation powered by the Inflection-2.5 LLM.

Built by Inflection AI, Pi (Personal Intelligence) prioritizes emotional intelligence and natural dialogue over raw task execution. It leverages the Inflection-2.5 model to achieve near-GPT-4 performance levels while maintaining a distinct, friendly persona across platforms like WhatsApp, SMS, and iOS. By focusing on high-EQ interactions and real-time web browsing, Pi serves as a digital confidant that remembers context to provide personalized, non-judgmental advice.

https://inflection.ai

View projects
Anthropic Messages API

The primary interface for building Claude-powered applications via structured, multi-turn message arrays.

Anthropic's Messages API serves as the gateway to Claude 3.5 Sonnet and the Opus model family. It replaces the legacy Text Completions endpoint with a robust, role-based structure (user vs. assistant) that supports multi-modal inputs like images and PDFs. Developers use this API to manage context windows up to 200k tokens, implement tool use (function calling), and stream real-time responses with sub-second time-to-first-token performance.

https://docs.anthropic.com/en/api/messages

View projects
Prompt caching

Prompt caching stores pre-computed token contexts to slash LLM API costs by up to 90% and response times by 80%.

Prompt caching optimizes LLM performance by storing the pre-computed KV (key-value) cache of repeated prompt prefixes (like system instructions, few-shot examples, or massive reference documents) directly in GPU memory. When a new request shares an identical prefix of 1024 tokens or more, the model skips the expensive prefill phase and reads directly from the cache. This simple structural shift reduces input token costs by up to 90% and slashes latency by up to 80% (making it a critical optimization for multi-turn chatbots and document analysis pipelines).

https://platform.openai.com/docs/guides/prompt-caching

View projects
TypeScript

TypeScript is an open-source superset of JavaScript: it adds static typing and compiles to clean, standards-based JavaScript.

TypeScript is a high-level, open-source language developed by Microsoft: it acts as a superset of JavaScript, adding a powerful static type system. This system enables compile-time type checking, catching errors before runtime (a critical benefit for large-scale applications). The TypeScript Compiler (TSC) reliably transpiles all code into clean, standards-based JavaScript (ES3 or newer), ensuring compatibility across any browser or host environment (Node.js, React.js, etc.).

https://www.typescriptlang.org/

View projects

pi-hydra: Mob Programming for Coding Agents

Related talks