B, 2-3 agents. The KV cache framing is the right one. Single-stream tok/s is what shows up in benchmarks but it's not what actually hurts when agents are sleeping between tool calls and waking up needing their full context. The question I'd want answered about an engine like this is how it handles partially-cold contexts, because agent sessions aren't uniform sustained reads, they're bursty and interleaved.

The output token limit and the context window are separate constraints. The context window is how much the model can see at once. The output limit is how much it can generate in a single API call.

In an agentic loop, each API call gets its own output budget. A 'turn' is one response from the model, whether that response contains a tool call, a reasoning step, or a final answer. So with a 1M context and a 64K output limit, the agent can run many turns where the context grows each round (accumulating tool results, prior thoughts, user messages), but each individual response is still capped at 64K tokens.

Expanding the output limit to 1M matters most for tasks that produce a lot in one shot, like writing a full document or a very long file. For most agentic workflows that naturally break into short turns, the per-call limit was rarely the bottleneck. The context window filling up was.

The streaming detection piece is what I'd want to understand better. How are you catching that the model is committing to a tool call before generation finishes? Pattern-matching on partial token sequences, or something at the structured output layer that gives you earlier signal?

On the caching claim, I want to make sure I'm reading this right.. Mutating at the head of a long context should bust prefix cache for everything after the insertion point. The win would come if the stable prefix before the insertion is long enough that most of the cache value lives there anyway, and tool invocations tend to cluster toward the end of a conversation. Is that the workload shape this is designed around?