convez.ai
MCP ServerAutomaçãoIntermediário
NevaMind-AI

memU — Engine de Memória para Agentes IA

NevaMind-AI/memU

Converta dados multimodais em memória estruturada para agentes IA com 10x menos tokens

Ver no GitHub
Muito bomPluginScore · de 0 a 100

Sistema de armazenamento estruturado que transforma dados brutos (chats, documentos, imagens, áudio, vídeo) em uma memória organizada e pronta para agentes IA navegarem. Reduz consumo de tokens em até 10x através de estruturação inteligente e recuperação contextual.

Instalação

$npx convez-ai NevaMind-AI/memU

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
15 mil
Forks
1,1 mil
Acompanhando
64
Saúde
62%
Último commit
há 1 dia
Versão
v1.5.1
Issues abertas
132
Contribuidores
33

Casos de uso

  • Agentes de IA com memória persistente e contextual de longo prazo
  • Sistemas de RAG otimizados com redução de overhead de tokens
  • Ingestão multimodal (texto, imagem, áudio, vídeo) em estrutura unificada

Veredito

memU é uma solução robusta para o desafio real de agentes IA manter contexto eficiente. O design em filesystem Markdown + embeddings é pragmático e bem documentado. Ideal para quem precisa de memória estruturada em produção, não apenas conceitual.

README

A tradução para o português deste README ainda está na fila.

MemU Banner

memU

Turn Raw Multimodal Data into Agent-Ready Structured Memory

PyPI version License: Apache 2.0 Python 3.13+ Discord Twitter

English | 中文 | 日本語 | 한국어 | Español | Français


memU is a data-to-memory engine for AI agents. It turns raw conversations, documents, images, audio, video, tool logs, and workspace files into an agent memory filesystem that agents can navigate, retrieve from, and use directly.

  • Raw in: chats, docs, URLs, images, audio/video, logs, and local workspaces
  • Structured out: index.md, memory.md, skill.md, topic subdocs, typed memory items, relations, and embeddings
  • Agent-ready: read the compact Markdown entrypoints, drill into subfiles, or load ranked context in one call

🔄 How It Works

Raw Multimodal Data → Agent Memory Filesystem → Agent Context

Raw Input                    memU Pipeline                 Filesystem Output
─────────────────────        ─────────────────────         ─────────────────────
chat logs                →   parse + segment           →   memory/preferences.md
documents / URLs         →   extract facts             →   index/api.md
images / video           →   caption + describe        →   memory/visual_context.md
audio                    →   transcribe + summarize    →   memory/events.md
tool logs                →   mine usage patterns       →   skill/tool_usage.md
workspace files          →   categorize + link         →   index/files.md
  1. Ingest — store each source as a Resource with its modality and source location
  2. Preprocess — parse text, caption images/video, transcribe audio, and normalize inputs
  3. Extract — turn raw content into typed MemoryItems such as profile, event, knowledge, behavior, skill, or tool memories
  4. Organize — categorize, cross-link, embed, and summarize memories into a browsable structure
  5. Write — emit compact Markdown entrypoints plus detailed subdocs under index/, memory/, and skill/
  6. Retrieve — return only the relevant context for the current user, agent, session, or task

🗂️ Agent Memory Filesystem

memU's primary output is a filesystem-like memory bundle for agents. The top-level files are compact entrypoints. The matching directories contain deeper subdocs that agents can open only when needed.

.memu/
├── index.md
├── memory.md
├── skill.md
├── index/
│   ├── architecture.md
│   ├── api.md
│   └── files.md
├── memory/
│   ├── decisions.md
│   ├── product_context.md
│   └── open_questions.md
└── skill/
    ├── testing.md
    ├── release.md
    └── coding_style.md
EntryRoleSubdocs
index.mdThe map of the workspace: what exists, where it lives, and how to navigate itindex/ holds deeper maps for architecture, APIs, modules, examples, and files
memory.mdThe compact long-term context an agent should load firstmemory/ holds decisions, constraints, product context, bugs, roadmap, and open questions
skill.mdThe operating manual for how to work in this projectskill/ holds repo-specific workflows for testing, release, migrations, coding style, and tool use

This gives agents a stable memory surface: they can start from three small files, then follow paths into focused Markdown documents instead of rereading the raw workspace every time.


🧩 What memU Builds

The Markdown filesystem is backed by structured memory records:

LayerWhat It RepresentsWhy Agents Use It
ResourceOriginal source artifact: conversation, document, image, video, audio, URL, or fileTrace memory back to its source
MemoryItemAtomic structured memory with a type and summaryInject precise facts, preferences, events, skills, and tool patterns
MemoryCategoryAuto-generated topic or folder with an evolving summaryLoad high-level context before drilling into details
CategoryItemRelationship between items and categoriesNavigate related memories without reprocessing the source
EmbeddingVector representation for resources, items, and categoriesRetrieve relevant context with low latency

Example memorize() output:

{
  "resource": {
    "id": "res_01",
    "url": "workspace/launch-meeting.mp4",
    "modality": "video",
    "caption": "A product planning discussion about onboarding and launch risks."
  },
  "items": [
    {
      "id": "mem_01",
      "memory_type": "event",
      "summary": "The team decided to simplify onboarding before the next launch review."
    },
    {
      "id": "mem_02",
      "memory_type": "profile",
      "summary": "The user prefers concise implementation plans with explicit verification steps."
    },
    {
      "id": "mem_03",
      "memory_type": "tool",
      "summary": "Use repository-wide search before editing configuration files to avoid missing duplicated settings."
    }
  ],
  "categories": [
    {
      "id": "cat_01",
      "name": "product_goals",
      "summary": "Current launch priorities, onboarding decisions, and unresolved risks."
    }
  ],
  "relations": [
    { "item_id": "mem_01", "category_id": "cat_01" }
  ]
}

Then an agent can call retrieve() to get a scoped, ranked context payload:

context = await service.retrieve(
    queries=[{"role": "user", "content": {"text": "What context matters for this launch task?"}}],
    where={"user_id": "123"},
    method="rag",
)

⭐️ Star the repository

If you find memU useful or interesting, a GitHub Star ⭐️ would be greatly appreciated.


✨ Core Features

CapabilityDescription
🗂️ Multimodal IngestionIngest conversations, documents, images, video, audio, URLs, logs, and workspace files
📁 Memory FilesystemProduce index.md, memory.md, skill.md, and focused subdocs under index/, memory/, and skill/
🧠 Typed Memory ExtractionExtract profile, event, knowledge, behavior, skill, and tool memories from raw sources
🧭 Automatic OrganizationBuild categories, relations, summaries, and embeddings without manual tagging
🤖 Agent-Ready RetrievalReturn scoped, ranked context that can be injected into any agent workflow
🧱 Pluggable StorageUse in-memory, SQLite, or Postgres backends with the same repository contracts
🔀 Profile-Based LLM RoutingRoute chat, embedding, vision, and transcription work through configurable LLM profiles

🎯 Use Cases

1. Conversation Memory

Turn chat logs into user preferences, goals, events, and relationship context.

await service.memorize(
    resource_url="conversations/user_123.json",
    modality="conversation",
    user={"user_id": "123"},
)

context = await service.retrieve(
    queries=[{"role": "user", "content": {"text": "What should I remember about this user?"}}],
    where={"user_id": "123"},
)

2. Workspace Context for Coding Agents

Convert docs, PR notes, logs, and design decisions into reusable project memory.

await service.memorize(resource_url="docs/architecture.md", modality="document")
await service.memorize(resource_url="examples/resources/logs/log1.txt", modality="document")

context = await service.retrieve(
    queries=[{"role": "user", "content": {"text": "How should I structure this module?"}}],
)

3. Multimodal Knowledge Layer

Extract searchable facts from documents, screenshots, images, videos, and audio notes.

await service.memorize(resource_url="research-notes.pdf", modality="document")
await service.memorize(resource_url="whiteboard.png", modality="image")
await service.memorize(resource_url="meeting-audio.mp3", modality="audio")

context = await service.retrieve(
    queries=[{"role": "user", "content": {"text": "What matters for the next research plan?"}}],
)

4. Tool and Agent Learning

Turn execution traces into tool memories that tell future agents when to use a tool and what mistakes to avoid.

await service.memorize(resource_url="agent_run.log", modality="document")

context = await service.retrieve(
    queries=[{"role": "user", "content": {"text": "Which tools worked for config editing?"}}],
)

🗂️ Architecture

memU's memory model is hierarchical enough for browsing and structured enough for direct retrieval:

LayerPrimary RoleRetrieval Role
ResourcePreserve source artifacts and captionsRecall original context when item/category summaries are not enough
ItemStore typed atomic memoriesLoad precise facts, events, preferences, skills, and tool patterns
CategoryMaintain topic-level summariesAssemble compact context for broad queries

See docs/architecture.md for the runtime view of MemoryService, workflow pipelines, storage backends, and LLM routing.


🚀 Quick Start

Option 1: Cloud Version

👉 memu.so — Hosted API for managed ingestion, structured memory, and retrieval

For enterprise deployment: info@nevamind.ai

Cloud API (v3)

Base URLhttps://api.memu.so
AuthAuthorization: Bearer <token>
MethodEndpointDescription
POST/api/v3/memory/memorizeIngest raw data and build structured memory
GET/api/v3/memory/memorize/status/{task_id}Check processing status
POST/api/v3/memory/categoriesList auto-generated categories
POST/api/v3/memory/retrieveQuery memory for agent context

📚 Full API Documentation


Option 2: Self-Hosted

Installation

pip install -e .

Requirements: Python 3.13+ and an OpenAI API key

Test with in-memory storage:

export OPENAI_API_KEY=your_key
cd tests && python test_inmemory.py

Test with PostgreSQL:

docker run -d --name memu-postgres \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=your_password \
  -e POSTGRES_DB=memu \
  -p 5432:5432 \
  pgvector/pgvector:pg16

export OPENAI_API_KEY=your_key
cd tests && python test_postgres.py

Custom LLM and Embedding Providers

from memu import MemUService

service = MemUService(
    llm_profiles={
        "default": {
            "base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
            "api_key": "your_key",
            "chat_model": "qwen3-max",
            "client_backend": "sdk"
        },
        "embedding": {
            "base_url": "https://api.voyageai.com/v1",
            "api_key": "your_key",
            "embed_model": "voyage-3.5-lite"
        }
    },
)

OpenRouter Integration

from memu import MemoryService

service = MemoryService(
    llm_profiles={
        "default": {
            "provider": "openrouter",
            "client_backend": "httpx",
            "base_url": "https://openrouter.ai",
            "api_key": "your_key",
            "chat_model": "anthropic/claude-3.5-sonnet",
            "embed_model": "openai/text-embedding-3-small",
        },
    },
    database_config={"metadata_store": {"provider": "inmemory"}},
)

📖 Core APIs

memorize() — Structure Raw Data

result = await service.memorize(
    resource_url="path/to/file.json",    # file path, URL, or directory
    modality="conversation",            # conversation | document | image | video | audio
    user={"user_id": "123"},            # optional: scope to a user or agent
)
# Returns immediately:
# { "resource": {...}, "items": [...], "categories": [...], "relations": [...] }
  • Converts raw input into typed memory items
  • Categorizes and embeds items without manual tagging
  • Preserves source resources and item-category relations

retrieve() — Load Agent Context

result = await service.retrieve(
    queries=[{"role": "user", "content": {"text": "What are their preferences?"}}],
    where={"user_id": "123"},   # scope filter
    method="rag"                # "rag" (fast) or "llm" (deep reasoning)
)
# Returns:
# { "categories": [...], "items": [...], "resources": [...], "next_step_query": "..." }
MethodSpeedCostBest For
rag⚡ msembedding onlyreal-time agent context
llm🐢 secondsLLM inferencedeeper semantic ranking

💡 Example Workflows

Always-Learning Assistant

export OPENAI_API_KEY=your_key
python examples/example_1_conversation_memory.py

Automatically extracts preferences, builds relationship models, and surfaces relevant context in future conversations.

Self-Improving Agent

python examples/example_2_skill_extraction.py

Monitors agent actions, identifies patterns in successes and failures, auto-generates skill guides from experience.

Multimodal Context Builder

python examples/example_3_multimodal_memory.py

Cross-references text, images, and documents automatically into a unified memory layer.


📊 Performance

memU achieves 92.09% average accuracy on the Locomo benchmark across all reasoning tasks.

View detailed results: memU-experiment


🧩 Ecosystem

RepositoryDescription
memUCore data-to-memory engine — ingestion, extraction, retrieval
memU-serverBackend with real-time sync and webhook triggers
memU-uiVisual dashboard for browsing and monitoring memory

Quick Links:


🤝 Partners


🤝 Contributing

# Fork and clone
git clone https://github.com/YOUR_USERNAME/memU.git
cd memU

# Install dev dependencies
make install

# Run quality checks before submitting
make check

See CONTRIBUTING.md for full guidelines.

Prerequisites: Python 3.13+, uv, Git


📄 License

Apache License 2.0


🌍 Community


⭐ Star us on GitHub to get notified about new releases!

Ver o original no GitHub

Bom para

Relacionados