{"id":419,"date":"2026-02-15T04:00:00","date_gmt":"2026-02-14T20:00:00","guid":{"rendered":"https:\/\/connectword.dpdns.org\/?p=419"},"modified":"2026-02-15T04:00:00","modified_gmt":"2026-02-14T20:00:00","slug":"how-to-build-a-self-organizing-agent-memory-system-for-long-term-ai-reasoning","status":"publish","type":"post","link":"https:\/\/connectword.dpdns.org\/?p=419","title":{"rendered":"How to Build a Self-Organizing Agent Memory System for Long-Term AI Reasoning\u00a0"},"content":{"rendered":"

In this tutorial, we build a self-organizing memory system for an agent that goes beyond storing raw conversation history and instead structures interactions into persistent, meaningful knowledge units. We design the system so that reasoning and memory management are clearly separated, allowing a dedicated component to extract, compress, and organize information. At the same time, the main agent focuses on responding to the user. We use structured storage with SQLite, scene-based grouping, and summary consolidation, and we show how an agent can maintain useful context over long horizons without relying on opaque vector-only retrieval.<\/p>\n

\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
import sqlite3\nimport json\nimport re\nfrom datetime import datetime\nfrom typing import List, Dict\nfrom getpass import getpass\nfrom openai import OpenAI\n\n\nOPENAI_API_KEY = getpass(\"Enter your OpenAI API key: \").strip()\nclient = OpenAI(api_key=OPENAI_API_KEY)\n\n\ndef llm(prompt, temperature=0.1, max_tokens=500):\n   return client.chat.completions.create(\n       model=\"gpt-4o-mini\",\n       messages=[{\"role\": \"user\", \"content\": prompt}],\n       temperature=temperature,\n       max_tokens=max_tokens\n   ).choices[0].message.content.strip()<\/code><\/pre>\n<\/div>\n<\/div>\n
We set up the core runtime by importing all required libraries and securely collecting the API key at execution time. We initialize the language model client and define a single helper function that standardizes all model calls. We ensure that every downstream component relies on this shared interface for consistent generation behavior.<\/p>\n
\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
class MemoryDB:\n   def __init__(self):\n       self.db = sqlite3.connect(\":memory:\")\n       self.db.row_factory = sqlite3.Row\n       self._init_schema()\n\n\n   def _init_schema(self):\n       self.db.execute(\"\"\"\n       CREATE TABLE mem_cells (\n           id INTEGER PRIMARY KEY,\n           scene TEXT,\n           cell_type TEXT,\n           salience REAL,\n           content TEXT,\n           created_at TEXT\n       )\n       \"\"\")\n\n\n       self.db.execute(\"\"\"\n       CREATE TABLE mem_scenes (\n           scene TEXT PRIMARY KEY,\n           summary TEXT,\n           updated_at TEXT\n       )\n       \"\"\")\n\n\n       self.db.execute(\"\"\"\n       CREATE VIRTUAL TABLE mem_cells_fts\n       USING fts5(content, scene, cell_type)\n       \"\"\")\n\n\n   def insert_cell(self, cell):\n       self.db.execute(\n           \"INSERT INTO mem_cells VALUES(NULL,?,?,?,?,?)\",\n           (\n               cell[\"scene\"],\n               cell[\"cell_type\"],\n               cell[\"salience\"],\n               json.dumps(cell[\"content\"]),\n               datetime.utcnow().isoformat()\n           )\n       )\n       self.db.execute(\n           \"INSERT INTO mem_cells_fts VALUES(?,?,?)\",\n           (\n               json.dumps(cell[\"content\"]),\n               cell[\"scene\"],\n               cell[\"cell_type\"]\n           )\n       )\n       self.db.commit()<\/code><\/pre>\n<\/div>\n<\/div>\n
We define a structured memory database that persists information across interactions. We create tables for atomic memory units, higher-level scenes, and a full-text search index to enable symbolic retrieval. We also implement the logic to insert new memory entries in a normalized and queryable form.<\/p>\n
\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
 def get_scene(self, scene):\n       return self.db.execute(\n           \"SELECT * FROM mem_scenes WHERE scene=?\", (scene,)\n       ).fetchone()\n\n\n   def upsert_scene(self, scene, summary):\n       self.db.execute(\"\"\"\n       INSERT INTO mem_scenes VALUES(?,?,?)\n       ON CONFLICT(scene) DO UPDATE SET\n           summary=excluded.summary,\n           updated_at=excluded.updated_at\n       \"\"\", (scene, summary, datetime.utcnow().isoformat()))\n       self.db.commit()\n\n\n   def retrieve_scene_context(self, query, limit=6):\n       tokens = re.findall(r\"[a-zA-Z0-9]+\", query)\n       if not tokens:\n           return []\n\n\n       fts_query = \" OR \".join(tokens)\n\n\n       rows = self.db.execute(\"\"\"\n       SELECT scene, content FROM mem_cells_fts\n       WHERE mem_cells_fts MATCH ?\n       LIMIT ?\n       \"\"\", (fts_query, limit)).fetchall()\n\n\n       if not rows:\n           rows = self.db.execute(\"\"\"\n           SELECT scene, content FROM mem_cells\n           ORDER BY salience DESC\n           LIMIT ?\n           \"\"\", (limit,)).fetchall()\n\n\n       return rows\n\n\n   def retrieve_scene_summary(self, scene):\n       row = self.get_scene(scene)\n       return row[\"summary\"] if row else \"\"<\/code><\/pre>\n<\/div>\n<\/div>\n
We focus on memory retrieval and scene maintenance logic. We implement safe full-text search by sanitizing user queries and adding a fallback strategy when no lexical matches are found. We also expose helper methods to fetch consolidated scene summaries for long-horizon context building.<\/p>\n
\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
class MemoryManager:\n   def __init__(self, db: MemoryDB):\n       self.db = db\n\n\n   def extract_cells(self, user, assistant) -> List[Dict]:\n       prompt = f\"\"\"\nConvert this interaction into structured memory cells.\n\n\nReturn JSON array with objects containing:\n- scene\n- cell_type (fact, plan, preference, decision, task, risk)\n- salience (0-1)\n- content (compressed, factual)\n\n\nUser: {user}\nAssistant: {assistant}\n\"\"\"\n       raw = llm(prompt)\n       raw = re.sub(r\"```json|```\", \"\", raw)\n\n\n       try:\n           cells = json.loads(raw)\n           return cells if isinstance(cells, list) else []\n       except Exception:\n           return []\n\n\n   def consolidate_scene(self, scene):\n       rows = self.db.db.execute(\n           \"SELECT content FROM mem_cells WHERE scene=? ORDER BY salience DESC\",\n           (scene,)\n       ).fetchall()\n\n\n       if not rows:\n           return\n\n\n       cells = [json.loads(r[\"content\"]) for r in rows]\n\n\n       prompt = f\"\"\"\nSummarize this memory scene in under 100 words.\nKeep it stable and reusable for future reasoning.\n\n\nCells:\n{cells}\n\"\"\"\n       summary = llm(prompt, temperature=0.05)\n       self.db.upsert_scene(scene, summary)\n\n\n   def update(self, user, assistant):\n       cells = self.extract_cells(user, assistant)\n\n\n       for cell in cells:\n           self.db.insert_cell(cell)\n\n\n       for scene in set(c[\"scene\"] for c in cells):\n           self.consolidate_scene(scene)<\/code><\/pre>\n<\/div>\n<\/div>\n
We implement the dedicated memory management component responsible for structuring experience. We extract compact memory representations from interactions, store them, and periodically consolidate them into stable scene summaries. We ensure that memory evolves incrementally without interfering with the agent\u2019s response flow.<\/p>\n
\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
class WorkerAgent:\n   def __init__(self, db: MemoryDB, mem_manager: MemoryManager):\n       self.db = db\n       self.mem_manager = mem_manager\n\n\n   def answer(self, user_input):\n       recalled = self.db.retrieve_scene_context(user_input)\n       scenes = set(r[\"scene\"] for r in recalled)\n\n\n       summaries = \"n\".join(\n           f\"[{scene}]n{self.db.retrieve_scene_summary(scene)}\"\n           for scene in scenes\n       )\n\n\n       prompt = f\"\"\"\nYou are an intelligent agent with long-term memory.\n\n\nRelevant memory:\n{summaries}\n\n\nUser: {user_input}\n\"\"\"\n       assistant_reply = llm(prompt)\n       self.mem_manager.update(user_input, assistant_reply)\n       return assistant_reply\n\n\n\n\ndb = MemoryDB()\nmemory_manager = MemoryManager(db)\nagent = WorkerAgent(db, memory_manager)\n\n\nprint(agent.answer(\"We are building an agent that remembers projects long term.\"))\nprint(agent.answer(\"It should organize conversations into topics automatically.\"))\nprint(agent.answer(\"This memory system should support future reasoning.\"))\n\n\nfor row in db.db.execute(\"SELECT * FROM mem_scenes\"):\n   print(dict(row))<\/code><\/pre>\n<\/div>\n<\/div>\n
We define the worker agent that performs reasoning while remaining memory-aware. We retrieve relevant scenes, assemble contextual summaries, and generate responses grounded in long-term knowledge. We then close the loop by passing the interaction back to the memory manager so the system continuously improves over time.<\/p>\n
In this tutorial, we demonstrated how an agent can actively curate its own memory and turn past interactions into stable, reusable knowledge rather than ephemeral chat logs. We enabled memory to evolve through consolidation and selective recall, which supports more consistent and grounded reasoning across sessions. This approach provides a practical foundation for building long-lived agentic systems, and it can be naturally extended with mechanisms for forgetting, richer relational memory, or graph-based orchestration as the system grows in complexity.<\/p>\n
\n
Check out the\u00a0Full Codes<\/a>.\u00a0<\/strong>Also,\u00a0feel free to follow us on\u00a0Twitter<\/mark><\/a><\/strong>\u00a0and don\u2019t forget to join our\u00a0100k+ ML SubReddit<\/a><\/strong>\u00a0and Subscribe to\u00a0our Newsletter<\/a><\/strong>. Wait! are you on telegram?\u00a0now you can join us on telegram as well.<\/a><\/strong><\/p>\n
The post How to Build a Self-Organizing Agent Memory System for Long-Term AI Reasoning\u00a0<\/a> appeared first on MarkTechPost<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"
In this tutorial, we build a s…<\/p>\n","protected":false},"author":1,"featured_media":29,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-419","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts\/419","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=419"}],"version-history":[{"count":0,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts\/419\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/media\/29"}],"wp:attachment":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=419"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=419"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=419"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}