How to Use Lorebook Token Budget Without Sacrificing Chat History

A well-built character knows things. A detective knows the city's underworld. A mage knows the rules of the arcane system they operate within. A brand representative knows the products they support. That knowledge does not belong in the system prompt — it belongs in a lorebook, where it is retrieved on demand rather than occupying the context window at all times.

The Blueprint Editor's "Generate Lorebook from Background" button does the initial extraction work for you: it reads the character's background data and produces a structured set of lorebook entries covering the world-building elements embedded in that text. This article covers what the generation actually does, what it produces, and how to calibrate the result.

What a Lorebook Is

Before the generation workflow: a lorebook is a collection of entries that inject contextual knowledge into a conversation when the user mentions specific keywords. The injection happens automatically, behind the scenes, without the user seeing it.

Each entry has:

• Title — the name of this lore element
• Content — a description of it (2–3 sentences with enough context for the model to use)
• Keywords — 2–5 trigger words or phrases that activate this entry
• Priority — determines which entries are included when the budget is tight

When a user's message contains one of an entry's keywords, the entry's content is injected into the context before the model generates a response. The model reads the injected content as [WORLD INFORMATION - Use this knowledge about the world] — separate from the conversation history — and uses it to answer with appropriate world-awareness.

Lorebook content stays in memory only when it is relevant. A 500-entry lorebook does not bloat every response; only the entries whose keywords appear in the recent conversation get injected. This is the core architectural difference between a lorebook and a system prompt: the system prompt is always present, the lorebook is conditional.

What the Generator Reads

The generator is in the World Lore section of the Blueprint Editor — the section labeled "Optional" at the bottom of the editor. When you click Generate Lorebook from Background, the backend reads the following character data:

• Character name
• Description (compiled from Identity and Psychology sections)
• Scenario (compiled from Background world + current situation + scene setup)
• Personality (compiled from Identity core traits and Friction)
• Blueprint background fields directly:
  • Origin (from Background section)
  • World (from Background section)
  • Current Situation (from Background section)

This is why the generator produces better results when the Background section is fully filled out. A character with a rich World field, a specific Current Situation, and a detailed Origin gives the generator substantially more to extract than a character with minimal background text.

The character must be saved before generating. The generator requires a saved characterId to query the database. If you have unsaved changes in the editor, save the character first.

What the Generator Produces

The generator calls meganova-ai/manta-flash-1.0 (fast model, optimized for structured output) at temperature 0.4. It extracts 6 entries by default — each entry covering one distinct world-building element across five categories:

The generator names the resulting lorebook "[Character Name] - World Lore" and attaches it to the character automatically. Each entry is created with enabled: true and a default priority of 10.

Note: Generating a new lorebook detaches any previously attached lorebook from the character. The previous lorebook is not deleted — it still exists in the Lore tab — but the character will use the newly generated one. If you want to preserve an existing lorebook, manage attachments manually in the Lore tab rather than re-generating.

How to Run It

1. Fill out the Background section — particularly World, Current Situation, and Origin
2. Save the character
3. Open the World Lore section in the Blueprint Editor
4. Click Generate Lorebook from Background
5. Wait for the spinner — the AI call takes a few seconds
6. A success toast confirms: Created lorebook "[Name] - World Lore" with [N] entries
7. The newly attached lorebook appears in the Lorebook panel with entry count

After generation, go to the Lore tab (in CharacterStudio) to review and edit the individual entries. The generated entries are a starting point, not a final product — they should be reviewed for accuracy and expanded where the generator did not go deep enough.

Editing Generated Entries

In the Lore tab, each generated entry shows its title, content, and keywords. The most common edits after auto-generation:

Add keywords the generator missed. The generator chooses keywords based on how an element appears in the character's background text. If the world refers to a location as "the capital" throughout the background, the generated keyword might be "capital" — but users are likely to type the actual name. Add the proper noun.

Expand entries that are too thin. The generator targets 2–3 sentences per entry. For entries the character will reference often, expand the content to include more operational detail — not everything about the concept, but the specific aspects the character would know and draw on.

Add entries the generator missed entirely. The generator extracts what is explicitly mentioned in the background text. It does not invent world-building that is absent. If the character's world has institutions, relationships, or history that are implied but not stated, write those entries manually.

Disable entries that are not useful. If the generator produced an entry for something that appears once in the background and has no conversational relevance, disable it (toggle in the Lore tab). Disabled entries are never injected regardless of keyword matches.

The Token Budget

The World Lore section has a Context Budget slider. This controls how much of the context window is allocated to lorebook content.

The slider runs from 0% to 50%, in steps of 5. The default is 25%.

The context window reference size for the budget calculation is 4096 tokens. At the default 25% budget:

• ~1024 tokens available for lorebook entries
• ~2572 tokens available for conversation history (the remainder after reserving ~500 tokens for the model's response)

The display shows these estimates live as you adjust the slider.

Higher budget: More lorebook entries can be injected per turn. Useful for lore-dense characters — worlds with complex systems, many factions, deep histories — where users are likely to touch multiple knowledge domains in a single conversation.

Lower budget: More conversation history is retained. Useful for characters in simpler settings where the lorebook is a supplement rather than a primary knowledge source, or for characters with long-running conversations where memory of earlier exchanges matters more than world knowledge injection.

The budget setting is stored in the Blueprint's lorebook configuration as budgetPercent. The backend reads this value directly when determining how many entries to inject for each turn.

How Injection Works During Chat

When the character is deployed and a user sends a message, the lorebook service:

1. Takes the last 10 messages from the conversation history
2. Concatenates their text and converts to lowercase
3. Checks each enabled lorebook entry: does any of its keywords appear as a substring in that text?
4. Collects all matching entries, sorted by priority
5. Injects matching entries up to the token budget
6. Adds the injected block to the context before the model generates a response

Keyword matching is case-insensitive and substring-based. A keyword of "ancient ruins" matches any message containing that phrase regardless of capitalization. A keyword of "ruin" would match "ruins," "ruined city," or "ruin your life." Specificity in keywords matters — broad keywords trigger entries too often; narrow keywords may never trigger.

Semantic Search

For lorebooks with many entries, MegaNova supports semantic search via vector embeddings. This is a separate capability from the keyword-based injection.

To enable it, go to the Lore tab and embed the lorebook entries. The embed endpoint processes each entry through a text embedding model and stores the vector. Once entries are embedded, semantic search can find relevant entries by meaning rather than exact keyword match — a query about "the war that ended the empire" could surface an entry titled "The Collapse of the Old Dominion" even if the specific phrase does not appear in the entry's keywords.

Embedding status is tracked per lorebook: total entries, embedded entries, and pending entries. Use "Embed All" to process the entire lorebook in one operation.

Semantic search is available as a standalone query feature in the Lore tab (for development and testing). Keyword-based injection during live chat is the primary delivery mechanism for lorebook content; semantic search is a supplementary tool for finding relevant entries manually.

Common Mistakes

Generating before the Background section is complete. The generator can only extract what the background text contains. Generating from a minimal background produces generic entries that do not capture anything specific about the character's world. Fill out Background fully first, then generate.

Using the generated lorebook as-is without review. The generator produces structurally correct entries but cannot know which world elements are most important to the character's story. Review every entry, strengthen the ones that matter, and add what is missing.

Setting the budget too high. A high lorebook budget increases per-turn context consumption but reduces how much conversation history the model can see. For characters in lore-heavy worlds, 25–35% is the practical ceiling for most deployments. Test at your chosen setting with realistic conversation lengths.

Generating when an existing lorebook should be kept. The generate button detaches the current lorebook and replaces it. If you want to add to an existing lorebook rather than replace it, add entries manually in the Lore tab instead of re-running generation.

Broad keywords that over-trigger. A keyword like "magic" in a fantasy setting will match nearly every conversation. Reserve that entry for the specific aspect of the magic system the character knows, and use specific enough keywords that the entry activates when relevant, not constantly.

Open the Blueprint Editor and generate your lorebook →

Stay Connected

💻 Website: Meganova Studio

🎮 Discord: Join our Discord

👽 Reddit: r/MegaNovaAI

🐦 Twitter: @meganovaai