Assistant — Compact model — Mediator

Assistant — Compact model

Separate model for history summarization. When the conversation grows past 20 turns / 20 000 characters, this model compresses the older part into a 4-8 bullet summary. Pick something cheaper here.

When the Assistant conversation grows too long, a separate call condenses the older part into a short summary — and only that summary plus the last 6 turns get sent to the AI on the next call. This lets a session run for hours without hitting the context window.

When does it run?

Automatically, before every send, if either condition is true: - ≥ 20 active turns (superseded ones don't count), - OR ≥ 20 000 characters of active content.

What it does

  1. Keeps the last 6 turns as-is.
  2. Compresses everything older into a 4-8 bullet summary.
  3. Marks the folded rows with superseded_by_id (not deleted — they stay in the DB, just skipped in the next prompt).
  4. Pins the summary as a 🗒 Conversation summary message at the top of the visible history. It stays there through subsequent compaction rounds.

Why a separate model?

Summarization is a narrow task, runs at low temperature (0.1), and a cheaper model is generally more than enough. The cost difference between chat and compact models can be an order of magnitude.

  • Same as the chat model (default): simplest, no separate thinking, but every compaction goes through the big model.
  • Something cheaper: e.g. Anthropic Haiku, OpenAI gpt-4o-mini, Groq Llama — perfect at summaries, a fraction of the cost.

If it fails

If the compact call errors, the send does not stop — it just logs a warning and proceeds with the full history. The summary will be retried on the next turn.