BetterThanTOON - Benchmark GCF vs TOON
blackwell-systems/betterthantoon
Benchmark técnico: GCF vence TOON em compreensão, geração e tokens
Ver no GitHubRepositório com dados de benchmark completos comparando GCF e TOON em compressão de dados tabulares. Inclui métricas de compreensão, geração e eficiência de tokens em múltiplos modelos (Claude, GPT, Gemini), demonstrando superioridade técnica do GCF em leitura, escrita e compactação de dados.
Instalação
npx convez-ai blackwell-systems/betterthantoon1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Comparar formatos de serialização de dados tabulares para LLMs
- Avaliar eficiência de tokens em diferentes estruturas de dados
- Validar confiabilidade de formato em geração de modelos
- Fundamentar decisões técnicas sobre compressão de dados
Veredito

Análise técnica rigorosa com dados reproduzíveis mostrando falhas sistêmicas do TOON em geração (0/5 em múltiplos modelos) e compreensão. GCF demonstra vantagem consistente, mas o valor prático depende de adequação ao caso específico de uso — não é conclusivo para todos os cenários tabulares.
README
A tradução para o português deste README ainda está na fila.
betterthantoon.com
GCF is better than TOON on every dimension. Their datasets, their tokenizer, their methodology. Full benchmark data proving it.
Why TOON fails
TOON claims ~40% token savings over JSON on tabular data. What they don't measure: whether LLMs can actually read it at scale, or produce valid output in it.
- Comprehension: TOON averages 68.5% at 500 records. GCF averages 90.7%.
- Generation: TOON's own decoder rejects LLM-generated output on 7 of 9 models. The error is always the same:
toon: cannot assign string to int. The model writes "target" where TOON expects the integer0. - Token efficiency: GCF wins all 6 datasets on TOON's own benchmark.
TOON's flat tabular design forces two failure modes that GCF eliminates structurally:
- Distance grouping failure: without section headers, models must scan 500 rows and filter by column value. They guess round numbers instead. TOON median error: 53.
- Semantic-to-integer encoding: TOON encodes categories as integers with no structural cue. Models can't perform this mapping unprompted.
Token efficiency (TOON's own benchmark)
Using TOON's benchmark code, TOON's tokenizer (o200k_base), and TOON's datasets:
| Dataset | GCF | TOON | Result |
|---|---|---|---|
| Semi-uniform event logs | 108,158 | 154,032 | GCF 42% smaller |
| E-commerce orders | 61,593 | 73,246 | GCF 19% smaller |
| Deeply nested config | 616 | 618 | GCF 0.3% smaller |
| Employee records | 49,055 | 49,966 | GCF 2% smaller |
| Analytics time-series | 8,398 | 9,127 | GCF 8% smaller |
| GitHub repos | 8,576 | 8,744 | GCF 2% smaller |
GCF wins all 6. Including the flat tabular datasets TOON was designed for.
Comprehension (can the model read it?)
500 symbols, 200 edges, 13 extraction questions. No format instructions. No system prompt.
| Model | GCF | TOON | JSON |
|---|---|---|---|
| Claude Opus 4.6 | 96.2% | 84.6% | 73.1% |
| Claude Sonnet 4.6 | 100% | 73.1% | 53.8% |
| Claude Haiku 4.5 | 96.2% | 69.2% | 57.7% |
| GPT-5.5 | 84.1% | 67.7% | 45.8% |
| GPT-5.4 | 78.0% | 56.0% | 44.1% |
| GPT-5.4-mini | 71.8% | 64.1% | 54.2% |
| Gemini 2.5 Pro | 100% | 76.9% | 58.3% |
| Gemini 3.1 Pro | 100% | 76.9% | 46.2% |
| Gemini 3.5 Flash | 100% | 61.5% | 46.2% |
| Gemini 2.5 Flash | 80.6% | 54.6% | 57.0% |
23 runs, 10 models, 3 providers. GCF wins 22, ties 1, loses 0. Four models hit 100%.
Generation (can the model write it?)
3-line primer, validated through real decoders. Same data, same prompt structure.
| Model | GCF | TOON | JSON |
|---|---|---|---|
| Claude Opus 4.6 | 5/5 | 0/5 | 5/5 |
| Claude Sonnet 4.6 | 5/5 | 2-3/5 | 5/5 |
| Claude Haiku 4.5 | 5/5 | 1-3/5 | 5/5 |
| GPT-5.5 | 4-5/5 | 1-2/5 | 5/5 |
| GPT-5.4 | 5/5 | 0/5 | 5/5 |
| GPT-5.4-mini | 5/5 | 0/5 | 5/5 |
| Gemini 2.5 Pro | 5/5 | 1/5 | 5/5 |
| Gemini 3.1 Pro | 5/5 | 0/5 | 5/5 |
No model has ever been trained on GCF. Every frontier model produces valid output on first exposure. TOON has been published for months and fails on 7 of 9 models.
The TOON generation failure
Every TOON generation failure produces the same error:
INVALID: symbols: index 0: distance: toon: cannot assign string to int
The model writes target in the distance column. TOON expects 0. The model would need to know, unprompted, that "target" maps to 0, "related" maps to 1, "extended" maps to 2. No model does this because the format gives no structural cue.
GCF expresses distance through section placement: targets go in ## targets, related in ## related. No integer mapping required.
Even when TOON is hand-held (pre-encoded integers in the prompt), GCF output is still 28% smaller.
What GCF has that TOON doesn't
- Session deduplication: symbols from prior calls become bare references. 92.7% savings by the 5th call. TOON re-serializes everything every time.
- Delta encoding: 81.2% additional savings on re-queries. TOON has no equivalent.
- Hierarchical grouping: section headers eliminate the distance-filtering failures that plague TOON at scale.
- Local IDs: edges cost ~4 tokens each regardless of identifier length. TOON repeats full qualified names.
- Streaming: zero-buffering with trailer summary. TOON has no streaming support.
We opened a PR on TOON's repo
We added GCF as a formatter to TOON's official benchmark. 9 lines of code. The PR is public:
The fork with full results:
How to try GCF
pip install gcf-proxy
Wrap any MCP server. Zero code changes. Or use the libraries directly:
pip install gcf-python # Python
npm install @blackwell-systems/gcf # TypeScript
go get github.com/blackwell-systems/gcf-go # Go
cargo add gcf # Rust
Links
- Live site
- Full benchmarks
- GCF vs TOON (detailed)
- Full eval data
- GCF Specification
- GCF Proxy
- Playground (live three-way comparison)
- Whitepaper (DOI: 10.5281/zenodo.20579817)
- TOON benchmark fork
License
MIT - Dayna Blackwell