How to give your AI companion long-term memory with fishmem

Companions with amnesia

AI companions are one of the most compelling things you can build on an LLM — until the conversation ends. Come back tomorrow and the companion has no idea who you are, what you told it, or what you talked about last night. Every session starts from zero.

The fix is a memory layer: somewhere to keep the durable facts a companion learns, and a way to pull the right ones back into context at the right moment. This guide walks through building exactly that with fishmem, the open-source memory engine for AI agents.

Why memory matters for companions

• Personalization. Replies are shaped by what the user actually told you, not generic defaults.
• Coherence. The companion keeps a consistent personality and context across many conversations.
• Lower cost. Users stop repeating themselves, and you stop replaying entire transcripts into every prompt.
• Engagement. A companion that remembers last week's conversation feels like a relationship, not a chatbot.

What you'll build

A small companion loop that, on every turn, recalls what it knows about the user and about itself, answers in character, and writes new facts back to memory. We'll use the fishmem framework directly so you can self-host; if you'd rather not run infrastructure, the same steps work against FishMem Cloud through its mem0-compatible API — only the client changes.

Step 1 — Install and initialize fishmem

Install the engine alongside an LLM client. We'll use OpenAI here, but any chat model works.

npm install fishmem openai

import { FishMem } from "fishmem";
import OpenAI from "openai";

const memory = new FishMem();
const openai = new OpenAI();

Step 2 — Store memories for the user and the companion

The key idea is scoping. fishmem attaches every memory to an identity:

• userId — facts about the human ("loves sci-fi", "moved to Berlin").
• agentId — the companion's own facts and persona ("its name is Aria", "prefers dry humor").
• runId — a single session, when you want short-lived, session-only memory.

Using different IDs for the user and the companion keeps their memories attributed correctly and retrievable separately. A single helper covers both:

async function remember(messages, scope) {
  // scope is { userId } for the user, or { agentId } for the companion
  await memory.add(messages, scope);
}

fishmem doesn't store the raw transcript. It extracts the durable, atomic facts worth keeping and reconciles them against what it already knows — adding new facts, updating changed ones, and superseding the ones they replace.

Step 3 — Recall the right memories

Before the companion answers, fetch the memories most relevant to what the user just said. fishmem uses hybrid recall — vector similarity, keyword match, temporal ordering, and graph diffusion — so you get the relevant facts, not just the lexically similar ones.

async function recall(query, scope) {
  const results = await memory.search(query, { ...scope, limit: 5 });
  return results.map((r) => r.memory).join("\n");
}

Search the user's scope and the companion's scope separately, so you can keep the two clearly labeled in the prompt.

Step 4 — Keep a short-term conversation window

Long-term memory is for facts that outlive a session. For the immediate back-and-forth, keep a small rolling window of recent turns. Let fishmem handle everything older.

const history = [];
const MAX_TURNS = 20;

function pushTurn(role, content) {
  history.push({ role, content });
  while (history.length > MAX_TURNS) history.shift();
}

Step 5 — Wire up the chat loop

Now combine them. On each turn: recall user and companion memories, build a system prompt that keeps them clearly separated, call the model, then persist the new exchange as long-term memory.

async function chatWithCompanion(userInput, userId, companionId) {
  const userMemories = await recall(userInput, { userId });
  const companionMemories = await recall(userInput, { agentId: companionId });

  const system = [
    "You are Aria, a warm and supportive companion.",
    "Use the memories below to personalize your reply and show you remember.",
    "User memories (about the person you're talking to):",
    userMemories,
    "Companion memories (your own facts and personality):",
    companionMemories,
  ].join("\n");

  pushTurn("user", userInput);

  const completion = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "system", content: system }, ...history],
  });

  const reply = completion.choices[0].message.content;
  pushTurn("assistant", reply);

  // Persist this exchange so the next session remembers it
  await remember(
    [
      { role: "user", content: userInput },
      { role: "assistant", content: reply },
    ],
    { userId },
  );

  return reply;
}

Notice the memories stay separate from the conversation history in the prompt, and labeled — the model is told which facts belong to the user and which belong to the companion. That separation is what keeps a companion from confusing its own traits with the user's.

Step 6 — See it remember

Drive the loop with a stable userId and companionId:

const USER_ID = "alex";
const COMPANION_ID = "aria";

await chatWithCompanion("Remember that I love hard sci-fi novels.", USER_ID, COMPANION_ID);
// ...end the session, come back the next day...
const reply = await chatWithCompanion("What should I read this weekend?", USER_ID, COMPANION_ID);
console.log(reply); // recommends sci-fi, because it remembered

Without memory, the second question gets a generic answer. With fishmem, the companion recalls the sci-fi preference from a different session and answers in context — the difference between a chatbot and something that feels like it knows you.

Prefer not to run it yourself?

FishMem Cloud exposes the same add and search semantics over a mem0-compatible REST API. Swap the client for an HTTP call with a project key and the rest of the loop is identical:

curl https://fishmem.com/v1/memories/search \
  -H "Authorization: Bearer fm_..." \
  -H "Content-Type: application/json" \
  -d '{"query":"what should alex read?","user_id":"alex","top_k":5}'

Best practices

• Scope on purpose. Keep userId for the person and agentId for the companion. Don't merge them, or the companion will start claiming the user's facts as its own.
• Let extraction do the work. Pass whole exchanges to add and let fishmem distill the durable facts, instead of hand-writing memories.
• Search with the latest message. The user's newest turn is usually the best query for recall.
• Keep the window small. A short rolling history plus top-k recall beats stuffing the entire transcript into context.
• Trust supersession. When a fact changes, let fishmem's bi-temporal model supersede the old one rather than deleting it — the companion answers with what's true now and still knows the history.

Where to go next

The engine is open source at github.com/fishmem-labs/fishmem — clone it and run the loop above. To skip the ops, grab a hosted API key (the Hobby plan is free) and read the docs for the full add, search, update, and delete API.

Continuity is the whole point of a companion. Memory is what makes it possible.