Start

Home / Big ideas

Idea6-min read+5 pts

What Is Compaction? Why a Long AI Chat Can Forget Your Rules

With Josh Schultz ·

Answer. Compaction is what an AI tool does when a long chat gets too full to keep every word. It replaces the old conversation with a short summary, and a summary keeps the big picture but can drop small details. A rule you stated only in the chat can be lost, while a rule saved in a file is read again.

The picture

Diagram: a desk covered in a tall pile of papers, with a circled paper labeled "never email Red Creek without Dana's OK." An arrow leads to a clean desk with one short note labeled "summary," and the circled paper is missing from it.

The pile on the left is a long chat. Somewhere near the bottom is a rule you gave at the start. When the desk is full, someone writes one short note about the pile and clears it. On the right, the desk is clean, and the note covers the main points. The rule may or may not have made it into the note, and that is the risk.

Think of it like this

A shift supervisor leaves a handover note for the next shift. It cannot repeat everything that happened on the floor, so it covers the jobs in progress, the problems, and what is next. A small instruction, such as "don't ship that lot until Dana signs off," might be left out because it did not seem important to whoever wrote the note.

That is compaction. The AI is the supervisor writing the note, and it decides what matters. Most of the time it does well. But it can only keep what it judges to be important, and a rule that sounded small in hour one can look small again in hour three.

Why does it matter at work?

Long jobs are where compaction shows up. You are three hours into a day of work, the chat is long, and the helper starts doing something that you told it not to do at the start.

Here is how this looks at Mesquite Ridge Metal Works. Dana Ortiz is working with a helper on a pile of supplier replies. First thing in the morning she told it, "Draft only. Never contact Red Creek Steel Supply without my OK." The day goes on, the chat grows, and the tool compacts it. Dana's rule was in the chat and nowhere else. The summary mentions the drafts and the open POs, but not the rule, and the helper behaves as if it was never given.

Nobody did anything wrong, and the AI did not decide to ignore her. The sentence simply did not make it through the squeeze. Anthropic's own best practices page warns about this. A boundary you state in conversation, it says, is "still a model reading the transcript, and the sentence can get lost when the conversation compacts," so "anything that matters belongs" in a rule that is enforced.

This is a close cousin of what we covered in why AI fails on long tasks, and it is part of why AI does not remember past chats the way a person would.

How does it work?

There are three parts: the limit, the squeeze, and what survives it.

The limit. An AI can only hold so much at once, which is the idea behind the context window. Anthropic's documentation says the tool "compacts automatically as you approach the limit," so a full desk does not end your session. Many current models have a very large window, so running out is rare, but a long day of files and replies can still get there.

The squeeze. When it compacts, the tool replaces the conversation with a structured summary. You can also do it yourself on purpose. If you type /compact and add a short note about what to keep, such as "keep the supplier rules and the open POs," the summary keeps what you chose, instead of what the tool guesses matters. You can also summarize part of a chat from the rewind menu (see what a checkpoint is).

What survives. Here is the good news. The documentation lists what the tool reads again from disk after compaction, which includes your project's saved instructions file and the plan the helper wrote in plan mode. Those come back whole. Things that only lived in the chat are summarized with everything else. So the safe place for a standing rule is a saved file, not your memory of having said it.

Diagram: two paths. The top path, "Said once in the chat," goes through a squeeze and comes out smaller with a question mark. The bottom path, "Saved in a file," goes around the squeeze and comes out whole, marked "read again."

This is the same idea as keeping your rules in one place and stacking context in the right order. A saved file of instructions, such as the kind described in what a markdown file is, is read at the start and again after compaction. A rule that cannot have exceptions can go one step further and become a guard, as in what a hook is.

One more habit helps. When you switch to a different job, start a fresh chat. Anthropic recommends clearing the chat "between unrelated tasks," because old conversation crowds out what you need next and costs more on every message.

Where do people get it wrong?

These are the mistakes I see most.

  1. Stating a standing rule only in the chat. If the rule matters tomorrow too, put it in a saved file.
  2. Running one chat all day. Start fresh when the job changes, and bring the rules from the file.
  3. Never looking at what was kept. After a long session, ask the helper to list the rules it is following, and compare them to yours.
  4. Blaming the AI for being careless. The cause is a squeeze that dropped a sentence, so the fix is in how you set things up.

Try this in 2 minutes

This test asks AI to explain why a long chat can lose an early rule and where to keep rules instead. You can use any chat tool.

  1. Open a new chat and paste this:
I am not technical. In 5 plain sentences, explain why a very long chat with an AI can lose a rule I gave at the start, using the comparison of a desk that fills up so someone summarizes the pile and clears it. Then tell me in 2 sentences where to keep rules so they never get lost.
  1. Check that the reply tells you to keep rules in a saved file and not only in the chat.
  2. Write the one rule you would hate to see lost in a long chat, and decide where to save it.

A chat reply that explains a chat as a desk that fills up, with the oldest papers at the bottom, then says to keep rules in a permanent instructions file.

When I tried this, the reply said "the oldest papers, including your rule from the start, are at the bottom of the pile." It explained that "a summary keeps the big themes but often drops small details, so your rule may get shortened, blurred, or left out," and that afterward "the AI only sees the short note, not your original words, so it can't follow a rule it no longer has."

For the fix, it said to put rules in "a permanent instructions file" that "is reloaded fresh at the start of every session instead of living in the pile." It also suggested repeating the most important rule in a short reminder when a chat gets very long. That is a good backup habit.

Frequently asked questions

What is compaction in AI?

It is when an AI tool replaces a long conversation with a short summary to make room. The summary keeps the main points and can drop small details.

Why did the AI forget what I told it?

If you said it only in the chat, it may not have made it into the summary. A rule saved in an instructions file is read again after compaction.

Can I control what gets kept?

Yes. In Claude Code you can type /compact with a note about what to keep. If you have not set it up yet, see how to install Claude Code.

Should I start a new chat for each job?

Usually yes. A fresh chat keeps the AI focused and costs less, and your saved rules come with you.

Use it today

From my training: this site teaches that a rule worth keeping belongs in a saved file, because a chat is a desk that will eventually be cleared.

Ideas are worth 5 points. Guides are worth 10 and up.

Get your first win this week.

One minute. Three guides picked for your job.

Start my first win