convez.ai
MCP ServerIntegração & APIsIntermediário
mutablelogic

go-llm

mutablelogic/go-llm

Interface unificada para múltiplos provedores de LLM em Go

Ver no GitHub
InicialPluginScore · de 0 a 100

MCP server e SDK multi-provider em Go para interagir com Google Gemini, Anthropic Claude e Mistral através de CLI unificada, API REST ou SDK. Suporta chat stateful, embeddings, tool use, agentes, saída estruturada e bot Telegram integrado.

Instalação

$npx convez-ai mutablelogic/go-llm

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
14
Forks
2
Acompanhando
1
Saúde
37%
Último commit
há 1 dia
Versão
v1.7.1
Issues abertas
40
Contribuidores
2

Casos de uso

  • Integrar múltiplos provedores LLM em aplicações Go com interface consistente
  • Construir chatbots e aplicações conversacionais com estado persistente
  • Executar agentes e ferramentas customizadas através do Model Context Protocol
  • Expor APIs REST para interação com LLMs e gerenciamento de sessões

Veredito

Projeto sólido e bem-estruturado para arquitetos de Go que precisam abstrair múltiplos provedores LLM. Oferece abstrações reais (CLI, HTTP, MCP, bot), não apenas bindings. Excelente para quem quer evitar vendor lock-in e precisa de session management nativo.

README

A tradução para o português deste README ainda está na fila.

go-llm

Go Reference License

A multi-provider LLM tool and SDK for Go.

Interact with large language models from Google Gemini, Anthropic Claude, and Mistral through a unified CLI, HTTP API server, or Go SDK. Supports stateless completion (ask), stateful multi-turn conversations (chat), embeddings, tool use, MCP integration, and a Telegram bot.

Features

  • Multi-Provider: Unified interface across Google Gemini, Anthropic Claude, and Mistral
  • CLI: Interactive terminal UI chat with markdown rendering, single-shot ask, and embedding generation
  • HTTP API Server: REST endpoints for models, sessions, tools, chat, ask, and embeddings with SSE streaming
  • Session Management: Stateful multi-turn conversations with persistent storage (file-backed JSON)
  • Tool Use: Extensible tool framework with built-in integrations for Home Assistant, NewsAPI, and WeatherAPI
  • MCP Server: Model Context Protocol (stdio JSON-RPC 2.0) server exposing registered tools and prompts
  • Telegram Bot: Full-featured bot with per-conversation sessions, file/image attachments, and slash commands
  • Attachments: Send files, images, and URLs as context in chat and ask
  • Agents: Reusable, versioned task definitions with input/output schemas, Go templates, and tool selection
  • Structured Output: JSON schema-based structured output for ask and agents
  • Thinking/Reasoning: Extended thinking support with configurable budgets (Anthropic, Gemini)
  • OpenTelemetry: Distributed tracing support for the HTTP API

Quick Start

Server

The server provides the REST API for all providers and tools. Run it with Docker:

docker volume create go-llm
docker run -d --name go-llm \
  -v go-llm:/data -p 8085:8085 \
  -e GEMINI_API_KEY="your-key" \
  ghcr.io/mutablelogic/llm run

CLI

The standalone CLI can be downloaded from the releases page, or built locally with make llm. Point it at a running server:

export LLM_ADDR="localhost:8085"

# List providers and models
llm providers
llm models

# Single-shot completion
llm ask --model gemini-2.0-flash "Explain KV cache in two sentences"

# Interactive chat (launches terminal UI)
llm chat --model gemini-2.0-flash

# Single-shot chat with session persistence
llm chat --model gemini-2.0-flash "What is the capital of France?"

# Generate embeddings
llm embedding --model text-embedding-004 "Hello, world"

Telegram Bot

The Telegram bot runs as a sidecar to the server, providing chat access via Telegram. It requires a bot token from @BotFather:

docker run -d --name go-llm-telegram \
  -e LLM_ADDR="your-server:8085" \
  -e TELEGRAM_TOKEN="your-bot-token" \
  ghcr.io/mutablelogic/llm telegram --model gemini-2.0-flash

Server Configuration

Providers

Providers are sources of models and generation capabilities. Some providers may support additional features like thinking/reasoning, tool use and embeddings. API keys for access to provider models are configured on the server via flags or environment variables:

ProviderFlagEnv VariableModels
Google Gemini--gemini-api-keyGEMINI_API_KEYGemini 2.0 Flash, Flash Lite, embedding models, etc.
Anthropic Claude--anthropic-api-keyANTHROPIC_API_KEYClaude Sonnet, Haiku, Opus, etc.
Mistral--mistral-api-keyMISTRAL_API_KEYMistral Large, Small, embedding models, etc.
ELIZA--eliza(none)Mock provider based on Weizenbaum's 1966 chatbot; no API key needed (en, de, fr)

Tools

Tools are external functions that can be called by the agent during generation. They are registered with the agent and exposed via the API and MCP server. Tools can be configured with flags or environment variables as needed for authentication or connection details. The following tools are included as examples of how to build tool integrations with the SDK:

ToolFlagEnv VariableDescription
Home Assistant--ha-endpoint, --ha-tokenHA_ENDPOINT, HA_TOKENQuery and control smart home devices
NewsAPI--news-api-keyNEWS_API_KEYSearch news articles and sources
WeatherAPI--weather-api-keyWEATHER_API_KEYCurrent weather and forecasts

HTTP & TLS

FlagEnv VariableDefaultDescription
--http.addrLLM_ADDRlocalhost:8085HTTP listen address
--http.prefix—/apiHTTP path prefix
--http.timeout—15mHTTP read/write timeout
--http.origin—(same-origin)CORS origin (* to allow all)
--tls.name——TLS server name
--tls.cert——TLS certificate file
--tls.key——TLS key file

Observability

FlagEnv VariableDescription
--otel.endpointOTEL_EXPORTER_OTLP_ENDPOINTOpenTelemetry collector endpoint
--otel.headerOTEL_EXPORTER_OTLP_HEADERSOpenTelemetry collector headers
--otel.nameOTEL_SERVICE_NAMEOpenTelemetry service name

Sessions and defaults are stored in $XDG_CACHE_HOME (or system temp if unset).

CLI Commands

The llm command-line tool can connect to any running go-llm server (set LLM_ADDR or --http.addr to point at it) and provides commands for generating text, managing models and sessions, inspecting tools, and running the server itself.

Generate

CommandDescriptionExample
askStateless single-shot completionllm ask --model gemini-2.0-flash --file "*.go" "Summarize this source code"
chatStateful chat (terminal UI or single-shot)llm chat --model gemini-2.0-flash
embeddingGenerate embedding vectorsllm embedding --model text-embedding-004 "text"

Model

CommandDescriptionExample
providersList available providersllm providers
modelsList modelsllm models
modelGet model detailsllm model gemini-2.0-flash

Session

Sessions represent stateful conversations with context and memory. They can be managed via the CLI or API, and are persisted on the server for later retrieval.

CommandDescriptionExample
sessionsList sessionsllm sessions
sessionGet session detailsllm session <id>
create-sessionCreate a new sessionllm create-session gemini-2.0-flash
update-sessionUpdate session metadatallm update-session <id>
delete-sessionDelete a sessionllm delete-session <id>

Tool

CommandDescriptionExample
toolsList registered toolsllm tools
toolGet tool detailsllm tool <name>

Agent

Agents are reusable, versioned task definitions that bundle a model, system prompt, optional input/output JSON schemas, Go template, and tool list into a single unit. They are defined as markdown files with YAML front matter and managed via the CLI or API. See the Agent Definition Files documentation for the full file format, field reference, template syntax, and examples.

CommandDescriptionExample
agentsList agentsllm agents
agentGet agent detailsllm agent <name>
create-agentCreate agents from markdown filesllm create-agent agents/*.md
delete-agentDelete an agentllm delete-agent <name>
run-agentRun an agent (create session, chat, return result)llm run-agent summarizer --input '{"text":"..."}'

Running an Agent

# Create the agent from a file
llm create-agent agents/summarizer.md

# Run with JSON input
llm run-agent summarizer --input '{"text": "The quick brown fox..."}'

# Run with file attachments
llm run-agent caption --file photo.jpg

# Run with URL attachment
llm run-agent summarizer --url https://example.com/article

# Keep the session after completion (for inspection or follow-up)
llm run-agent summarizer --no-delete --input '{"text": "..."}'

The run-agent command creates a temporary session, sends the templated message, and returns the structured JSON result. By default the session is deleted after completion; use --no-delete to persist it.

Server

CommandDescriptionExample
runStart the serverllm run
telegramRun as a Telegram bot, requiring a serverllm telegram --model gemini-2.0-flash

Use llm --help or llm <command> --help for full options.

Interactive Chat

The chat command launches a terminal UI with markdown rendering, streaming responses, and slash commands:

Interactive Chat

Slash Commands

CommandDescription
/modelSwitch model
/modelsList available models
/providersList providers
/sessionShow current session
/sessionsList sessions
/nameRename current session
/labelSet session label
/systemSet system prompt
/thinkingToggle thinking/reasoning
/toolsToggle tool use
/agentsList agents
/agentShow agent details (running agents from chat is not yet supported)
/fileAttach a file
/urlAttach a URL
/resetReset conversation
/deleteDelete current session
/helpShow help

Development

Architecture

%%{init: {"flowchart": {"htmlLabels": false}} }%%
flowchart LR
    subgraph Clients["Clients"]
        direction TB
        CLI["`**cmd/llmserver**
        CLI Tool`"]
        TUI["`**pkg/ui/bubbletea**
        Terminal UI`"]
        SDK["`**pkg/httpclient**
        SDK Client`"]
        Telegram["`**pkg/ui/telegram**
        Telegram Bot`"]
    end

    subgraph Server["go-llm Server"]
        direction TB
        API["`**pkg/httphandler-new**
        REST API`"]
        Agent["`**pkg/llmmanager**
        Manager`"]
        AgentDef["`**pkg/agent**
        Agent Parsing`"]
        Sessions["`**pkg/store**
        Memory, File`"]
        Tools["`**pkg/tool**
        Tool Registry`"]
        MCP["`**pkg/mcp**
        MCP Server`"]
    end

    subgraph Providers["LLM Providers"]
        direction TB
        Gemini["`**pkg/provider/google**
        Google Gemini`"]
        Claude["`**pkg/provider/anthropic**
        Anthropic Claude`"]
        Mistral["`**pkg/provider/mistral**
        Mistral`"]
        Eliza["`**pkg/provider/eliza**
        ELIZA (mock)`"]
    end

    subgraph ToolImpls["Example Tools"]
        direction TB
        HA["`**pkg/homeassistant**
        Home Assistant`"]
        News["`**pkg/newsapi**
        NewsAPI`"]
        Weather["`**pkg/weatherapi**
        WeatherAPI`"]
    end

    CLI --> API
    TUI --> API
    SDK --> API
    Telegram --> API
    API --> Agent
    Agent --> AgentDef
    Agent --> Sessions
    Agent --> Tools
    Agent --> Gemini
    Agent --> Claude
    Agent --> Mistral
    Agent --> Eliza
    Tools --> HA
    Tools --> News
    Tools --> Weather
    MCP --> Tools

SDK Example

The Go SDK provides interfaces for building LLM-powered applications:

package main

import (
    "context"
    "fmt"
    "log"

    llm "github.com/mutablelogic/go-llm"
    "github.com/mutablelogic/go-llm/provider/google"
)

func main() {
    ctx := context.Background()

    // Create a provider
    client, err := google.New(ctx, "your-api-key")
    if err != nil {
        log.Fatal(err)
    }

    // Get a model
    model, err := client.GetModel(ctx, "gemini-2.0-flash")
    if err != nil {
        log.Fatal(err)
    }

    // Stateless completion
    gen := model.(llm.Generator)
    response, err := gen.WithoutSession(ctx, "Explain KV cache in two sentences")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(response)
}

Key Packages

PackageDescription
pkg/llmmanagerCentral manager orchestrating providers, sessions, and tools
pkg/agentAgent definition parsing (template execution, input validation, funcmap)
pkg/provider/{google,anthropic,mistral,eliza}Provider implementations
pkg/storeStorage backends for sessions and agents (in-memory, file-backed JSON)
pkg/toolTool interface and toolkit registry
pkg/schemaCore types (Model, Message, ContentBlock, Attachment, Session, etc.)
pkg/httpclientHTTP client for the go-llm API
pkg/httphandler-newHTTP handler layer for the REST API server
pkg/mcpModel Context Protocol server (stdio JSON-RPC 2.0)
pkg/uiChat UI abstraction (bubbletea terminal UI, Telegram bot, shared command handler)

Core Interfaces

InterfaceDescription
ClientProvider connection — Name(), ListModels(), GetModel()
GeneratorText generation — WithoutSession() (stateless), WithSession() (stateful)
EmbedderEmbedding generation — Embedding(), BatchEmbedding()
DownloaderModel management — DownloadModel(), DeleteModel()

Building

git clone https://github.com/mutablelogic/go-llm.git
cd go-llm
make

Build the standalone CLI:

make llm

Makefile Targets

TargetDescription
make allBuild all binaries
make llmBuild the primary CLI binary
make dockerBuild Docker image
make docker-pushPush Docker image to GHCR
make docker-versionPrint version tag
make testRun all tests
make unit-testRun unit tests
make coverage-testRun tests with coverage
make tidyRun go mod tidy
make cleanRemove build artifacts

Cross-compilation is supported via OS and ARCH variables:

OS=linux ARCH=arm64 make

Contributing & License

Please file issues and feature requests in GitHub Issues. Licensed under Apache 2.0.

Ver o original no GitHub

Bom para

Relacionados