How does an LLM remember earlier messages in a conversation?

Spoiler: it doesn't. The app re-sends the whole chat every time.

Picture it first

Imagine a friend who forgets everything every time you blink. To keep talking, you hand them a notebook with the whole conversation written down before every sentence. They read the notebook super fast and reply as if they remembered. If the notebook gets too thick, you start tearing out old pages — and then they really do forget those parts.

Now the same idea, in simple QA words

The AI itself has no memory at all. Every time you send a message, the chat app quietly sends the whole earlier conversation again along with your new message. The AI reads it all super fast and replies as if it remembered. If the conversation gets very long, the app removes or shortens old messages — and then the AI truly 'forgets' them. For QA: memory bugs are almost always in the app, not the AI.

How the story connects to the real thing

In the storyIn the AI system
Friend who forgets everything when they blinkThe AI model (no memory)
The notebook you hand them every timeChat history sent with every message
Tearing out old pages when the notebook is thickApp trims old messages
Friend can't recall a torn-out pageBot forgets your earlier detail

Real-world example

A customer tells a telecom chatbot in message 1: 'I'm on the prepaid ₹299 plan'. In message 9 they ask 'Can I upgrade?'. The bot only knows the plan because the app re-sent message 1. If the app trimmed history to the last 6 messages, the bot will ask again or — worse — assume a postpaid plan. That's a memory bug in the app, not in the model.

Step by step, in plain words

  1. You send a new message.
  2. The app attaches all earlier messages to it.
  3. The AI reads everything from the start.
  4. The AI replies.
  5. The app saves the reply in the history for next time.

How a tester tests it

  • Plant a fact early (name, plan, constraint) and ask about it 5, 10, 20 turns later.
  • Find the trimming boundary: at which turn does the bot lose the fact?
  • Test cross-session memory: is information from user A ever visible to user B? (privacy defect).
  • Test corrections: user says 'Actually I'm on ₹399' — does the bot use the newest fact?

Try it yourself

1) Tell a chatbot your favourite colour, chat about 10 unrelated things, then ask what your colour is. 2) In a new chat, ask again — note what persists and what doesn't.

Interview angle

“A chatbot forgets user details after long chats. How do you investigate?”

First I confirm the model is stateless and inspect what the app actually sends — the request payload. Usually history is trimmed by message count or token budget, or summarisation dropped the detail. I write a test that plants facts at known turns and binary-searches the turn where they disappear, then recommend pinning key facts into a structured memory block.