Prompt caching

Many requests start with the same tokens: a long system prompt, a document, an agent’s conversation history. When they do, we reuse the work already done for that prefix instead of computing it again. Those tokens are billed at the cached input price. For Kimi K3.1 Pro, that is a tenth of the normal input price.

Caching is automatic. There is no setting or parameter to turn on.

How it works

To read a prompt, the model computes an internal state for every token, called the KV cache. When the start of a new request matches a recent request from your account token for token, the model reuses that state and only processes the part that is new. Fewer tokens to process means a faster first token and a lower price.

• A cached prefix is kept for up to 1 hour after it was last used.
• The cache is scoped to your account. Other accounts never hit or see your cached prefixes.
• A cache miss is not an error. The request is processed normally and billed at the input price.

Getting more cache hits

• Put content that stays the same at the start of the prompt: system prompt, instructions, tool definitions, reference documents.
• Put content that changes, such as the user’s latest message, at the end.
• Keep the shared prefix byte-for-byte identical. A changed timestamp or reordered tool list near the start breaks the match for everything after it.

Seeing cached tokens

The usage object reports how many prompt tokens came from the cache:

"usage": {
  "prompt_tokens": 12840,
  "completion_tokens": 312,
  "total_tokens": 13152,
  "prompt_tokens_details": { "cached_tokens": 12288 }
}

Here 12,288 tokens are billed at the cached price and the remaining 552 at the normal input price.

Privacy

The KV cache is numbers computed by the model, not your prompt text, and it lives only in GPU memory. It is never written to disk and is dropped after at most 1 hour. Prompt caching is compatible with our Zero Data Retention Policy.