Why Your AI Roleplay Bot Forgets: Context Windows Explained

Forty messages into a great roleplay, your character forgets your name, the promise they made two scenes ago, or the fact that they hate the sea. It feels like the bot stopped paying attention. What actually happened is simpler: that part of the story no longer fits in what the model can see.

AI models don't remember a chat the way people do. Each reply is written from scratch, using only the text sent with that one request. The limit on how much text that can be is the context window, and once you understand it, most “memory” problems become easy to diagnose and fix.

Tokens and the context window

Models don't read words; they read tokens, small chunks of text that can be a whole word, part of a word or a punctuation mark. For English, a common rule of thumb is that one token is about three-quarters of a word, so 100 tokens is roughly 75 words (OpenAI's token guide). The exact count depends on the model, because each one splits text a little differently.

The context window is the maximum number of tokens a model can take in for a single reply, counting both what you send and what it writes back. Think of it as the model's desk: everything it needs for this reply has to fit on the desk at once, and anything that doesn't fit simply isn't there.

Windows vary enormously. Some chat models work with a few thousand tokens; newer models such as DeepSeek V4.1 Flash accept up to a million. But the window the model supports and the window your chat actually uses are two different numbers. Janitor AI has its own context size setting, and that setting is what decides how much of your story gets sent.

What fills the window, and what falls out first

On every message, Janitor AI builds one request from two kinds of content (Janitor's token guide):

• Permanent content, sent every time: the character's Personality and Scenario, your persona, Chat Memory and any Advanced Prompt.
• Temporary content, sent only while there's room: the chat history and the Example Dialogs.

Room also has to be left for the reply itself, which is what Max New Tokens reserves. Whatever space remains goes to history, newest messages first. When the chat grows past that space, the oldest messages are dropped. The model hasn't forgotten them in any human sense; they just weren't sent.

A worked example

Here's how a 16,000-token context might be spent. The numbers are illustrative, not Janitor defaults.

If each message averages around 250 tokens (roughly 190 words), about 50 messages of history fit. Message 51 pushes out message 1. That's why a bot can be perfectly consistent for an hour and then suddenly lose track of how the story began: nothing about the bot changed, the beginning just scrolled off the desk.

It also explains a few patterns players notice:

• Long replies shorten the bot's memory. Bigger messages mean fewer of them fit.
• A heavy card shortens it too. Every permanent token comes out of the history budget.
• Example dialogs stop working in long chats. They're temporary, so they leave with the oldest messages.

Why a bigger window isn't a complete fix

If forgetting is about space, the obvious fix is a model with a huge window and the context slider turned all the way up. That helps, but it comes with three trade-offs.

Cost. On a paid proxy you're billed for input tokens, and the whole context is input on every single message. Doubling the context roughly doubles the input cost of each reply. Some providers discount repeated input through caching (OpenAI's token guide describes cached tokens billed at a reduced rate), but you still pay for everything you send.

Speed. The model has to read the entire context before it writes the first word, so a longer context means a longer wait before the reply starts.

Attention. Fitting in the window isn't the same as being noticed. Research has found that language models use information at the start and end of a long prompt more reliably than information buried in the middle (Liu et al., “Lost in the Middle”). Newer models handle long contexts better than the ones in that study, but a detail mentioned once, 80,000 tokens back, is still more likely to be missed than a short line in the character card.

The practical takeaway: a large window is a good safety margin, not a substitute for putting important facts where the model will reliably see them.

How to make your bot remember more

Match the fix to what's being forgotten.

Use Chat Memory as a running summary

Chat Memory is permanent, so it's the best place for the handful of facts the story depends on: names, relationships, promises, injuries, where the characters are now. Keep it to short, factual lines and update it every few scenes. A 300-token summary of the first hundred messages does more for continuity than sending all hundred.

Set the context size on purpose

Raise Janitor's context size until the bot stops losing recent scenes, not all the way to the model's maximum. Each step up costs a little more per message and adds a little delay. If you use a long-context model through a proxy, check the limit your provider actually serves, which can be lower than the model's headline number.

Keep the card lean

Every permanent token comes out of the history budget. Janitor suggests most well-made bots work best with under 1,500 permanent tokens (token guide). Our character card guide shows how to write a personality that's short and still specific.

The short version

Your bot only knows what's sent with each message. The card and Chat Memory are always sent; the chat history is sent until it runs out of room, oldest first. So when a bot forgets, ask what was dropped, then either make room (a leaner card, a larger context) or move the important fact somewhere permanent.

If you want more room to work with, a long-context model helps. Our DeepSeek V4.1 Flash setup guide shows how to connect one to Janitor AI through MegaNova in a few minutes.

What’s Next?

Sign up and explore now.

🔍 Learn more: Visit our blog and documents for more insights or contact us for a demo of how MegaNova can power your AI products.

📬 Get in touch: Join our Discord community for help or Contact Us.