convez.ai
MCP ServerIntegração & APIsIntermediário
blackwell-systems

GCF Go — Formato Wire Eficiente para LLMs

blackwell-systems/gcf-go

Formato wire 53% mais eficiente que JSON para estruturas de dados em IA

Ver no GitHub
InicialPluginScore · de 0 a 100

Implementação em Go do GCF, um formato de serialização otimizado para LLMs que usa 25.5% menos tokens que TOON e 53% menos que JSON. Oferece 100% de compreensão em modelos frontier com suporte nativo a grafos de código, deduplicação de sessão e CLI para conversão sem código.

Instalação

$npx convez-ai blackwell-systems/gcf-go

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
8
Forks
0
Acompanhando
0
Saúde
42%
Último commit
há 1 mês
Versão
v1.8.0
Issues abertas
0
Contribuidores
1

Casos de uso

  • Reduzir custos de token em aplicações com LLMs
  • Serializar grafos de código e dependências para análise contextual
  • Converter estruturas complexas (JSON, mapas, structs) com máxima compressão

Veredito

Ferramenta sólida e bem testada (1.700+ avaliações LLM, 477M iterações fuzzing) para otimização de custos em pipelines de IA. Útil principalmente para quem trabalha com contexto largo e precisa reduzir consumo de tokens sem perder fidelidade estrutural.

README

A tradução para o português deste README ainda está na fila.

gcf-go

Go implementation of GCF — the most token-efficient wire format for LLMs. A drop-in alternative to JSON and TOON for any structured data.

100% comprehension on every frontier model tested. 25.5% fewer tokens than TOON, 53% fewer than JSON across 15 datasets. 90.7% on structurally complex code graphs (vs TOON 68.5%, JSON 53.6%). 1,700+ LLM evaluations. Zero training.

Docs: gcformat.com · Playground · GCF vs TOON

Install

go get github.com/blackwell-systems/gcf-go

Zero dependencies. Single package. Don't want to change code? Use the MCP proxy for zero-code adoption.

CLI

Standalone binaries are attached to each release. The CLI is optional; it's for converting files from the command line without writing code.

# Install
go install github.com/blackwell-systems/gcf-go/cmd/gcf@latest

# Or download a binary from the latest release

# Usage
gcf encode < payload.json    # JSON to GCF
gcf decode < payload.gcf     # GCF to JSON
gcf stats  < payload.json    # token comparison

Quick Start

import gcf "github.com/blackwell-systems/gcf-go"

data := map[string]any{
    "employees": []map[string]any{
        {"id": 1, "name": "Alice", "department": "Engineering", "salary": 95000},
        {"id": 2, "name": "Bob", "department": "Sales", "salary": 72000},
    },
}
output := gcf.EncodeGeneric(data)

Output:

## employees [2]{department,id,name,salary}
Engineering|1|Alice|95000
Sales|2|Bob|72000

Works on any Go value: maps, slices, structs. One header declares field names, rows are positional values.

Graph Profile

For code graph data with symbols, edges, and distance groups:

p := &gcf.Payload{
    Tool: "context_for_task", TokenBudget: 5000, TokensUsed: 1847,
    Symbols: []gcf.Symbol{
        {QualifiedName: "pkg.Auth", Kind: "function", Score: 0.78, Provenance: "lsp", Distance: 0},
        {QualifiedName: "pkg.Server", Kind: "function", Score: 0.54, Provenance: "lsp", Distance: 1},
    },
    Edges: []gcf.Edge{{Source: "pkg.Server", Target: "pkg.Auth", EdgeType: "calls"}},
}
output := gcf.Encode(p)

Output:

GCF tool=context_for_task budget=5000 tokens=1847 symbols=2 edges=1
## targets
@0 fn pkg.Auth 0.78 lsp
## related
@1 fn pkg.Server 0.54 lsp
## edges [1]
@0<@1 calls

Decode

p, err := gcf.Decode(input)
if err != nil {
    log.Fatal(err)
}
fmt.Println(p.Tool, len(p.Symbols), "symbols", len(p.Edges), "edges")

Session Deduplication

Track transmitted symbols across multiple tool responses. Previously-sent symbols become bare references instead of full declarations:

sess := gcf.NewSession()

out1 := gcf.EncodeWithSession(payload1, sess) // full declarations
out2 := gcf.EncodeWithSession(payload2, sess) // reused symbols as "@N  # previously transmitted"

By the 5th call in a session: 92.7% token savings vs JSON.

Streaming Encode

Write GCF output incrementally as symbols and edges arrive. Zero buffering, O(1) memory per row. Ideal for MCP servers that walk large graphs or paginate results:

enc := gcf.NewStreamEncoder(w, "context_for_task", gcf.StreamOptions{TokenBudget: 5000})

// Symbols emit immediately as they're discovered.
enc.WriteSymbol(gcf.Symbol{QualifiedName: "pkg.Auth", Kind: "function", Score: 0.95, Provenance: "lsp", Distance: 0})
enc.WriteSymbol(gcf.Symbol{QualifiedName: "pkg.Server", Kind: "function", Score: 0.60, Provenance: "lsp", Distance: 1})

// Edges emit immediately too.
enc.WriteEdge(gcf.Edge{Source: "pkg.Server", Target: "pkg.Auth", EdgeType: "calls"})

// Close emits the ## _summary trailer with final counts.
enc.Close()

Output:

GCF tool=context_for_task budget=5000
## targets
@0 fn pkg.Auth 0.95 lsp
## related
@1 fn pkg.Server 0.60 lsp
## edges [?]
@0<@1 calls
## _summary symbols=2 edges=1 sections=targets:1,related:1,edges:1

The [?] marker signals deferred count. The ## _summary trailer provides counts after the data. The LLM has both the data and the counts in context. Standard Decode() handles streaming output with no changes.

Delta Encoding

When the consumer already has a prior context pack, send only what changed:

delta := &gcf.DeltaPayload{
    Tool:     "context_for_task",
    BaseRoot: "aaa111",
    NewRoot:  "bbb222",
    Removed:  []gcf.Symbol{{QualifiedName: "pkg.OldFunc", Kind: "function"}},
    Added:    []gcf.Symbol{{QualifiedName: "pkg.NewFunc", Kind: "function", Score: 0.85, Provenance: "rwr"}},
    DeltaTokens: 30,
    FullTokens:  200,
}

output := gcf.EncodeDelta(delta)

81.2% savings on re-queries where the pack changed slightly.

Generic Encoding

Encode any Go value (not just graph payloads) into GCF tabular format:

data := map[string]any{
    "employees": []map[string]any{
        {"id": 1, "name": "Alice", "department": "Engineering", "salary": 95000},
        {"id": 2, "name": "Bob", "department": "Sales", "salary": 72000},
    },
}
output := gcf.EncodeGeneric(data)

Output:

## employees [2]{id,name,department,salary}
1|Alice|Engineering|95000
2|Bob|Sales|72000

Works on maps, slices, structs, and primitives. Arrays of uniform objects get tabular rows. Nested objects use ## key section headers.

API

FunctionDescription
Encode(p *Payload) stringEncode a graph payload to GCF text
EncodeGeneric(data any) stringEncode any value to GCF tabular format
Decode(input string) (*Payload, error)Parse GCF text back to a Payload
EncodeWithSession(p *Payload, s *Session) stringEncode with session deduplication
EncodeDelta(d *DeltaPayload) stringEncode a delta (added/removed only)
NewStreamEncoder(w, tool, opts) *StreamEncoderCreate a streaming encoder (zero-buffering)
NewSession() *SessionCreate a new session tracker (thread-safe)

Types

TypePurpose
PayloadFull GCF payload: tool, budget, symbols, edges, pack root
SymbolGraph node: qualified name, kind, score, provenance, distance
EdgeDirected relationship: source, target, edge type
DeltaPayloadDiff between two packs: added/removed symbols and edges
SessionThread-safe tracker for multi-call deduplication
StreamEncoderStreaming encoder: WriteSymbol, WriteEdge, WriteBareRef, Close
StreamOptionsConfig for streaming: TokenBudget, TokensUsed, PackRoot, Session
KindAbbrev / KindExpandBidirectional kind abbreviation maps

Benchmarks

1,700+ LLM evaluations across 10 models, 3 providers, and 51 independent test runs.

GCFTOONJSON
Comprehension (23 runs, 10 models)90.7%68.5%53.6%
Generation (28 runs, 9 models)5/51.0/55.0/5
Input tokens (500 symbols)11,09016,37853,341
Output tokens (100 symbols)5,9768,93716,121

GCF wins 13/15 datasets on the expanded token efficiency benchmark. Full results: gcformat.com/guide/benchmarks

Links

License

MIT - Dayna Blackwell

Ver o original no GitHub

Bom para

Relacionados