convez.ai
MCP Server
zakky8

TENET

zakky8/TENET

Ver no GitHub
BomPluginScore · de 0 a 100

TypeScript AI agent framework with multi-judge hallucination verifier + Vectara HHEM-2.1, pre-tool-call governance/approval/audit, WASM-sandboxed tools, MCP OAuth gateway. 10 surfaces (Discord/Slack/Telegram/Teams/web/REST/gRPC/ticketing). BENCHMARKS gated in CI on every PR. LangChain alternative. 0 CVEs. Apache-2.0.

Instalação

$npx convez-ai zakky8/TENET

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
0
Forks
0
Acompanhando
0
Saúde
100%
Último commit
há 4 meses
Versão
—
Issues abertas
0
Contribuidores
1

README

A tradução para o português deste README ainda está na fila.

TENET — Verification-First AI Agent Framework

Open-source TypeScript agent framework where hallucination defense, source-grounding, capability-token tool sandboxing, and platform rate-limit scheduling are first-class layers — not bolt-ons. One config powers Telegram, Discord, Slack, MS Teams, embedded web widget, REST + gRPC, and ticketing webhooks (Zendesk · Intercom · Freshdesk · ServiceNow). Every BENCHMARKS row is gated in hermetic CI.

CI License: Apache 2.0 Node 22+ TypeScript Strict Tests BENCHMARKS gated ES2024 Code of Conduct

Status: 16 shipped phases (P0–P15). 1036 tests across 83 suites, all green. Every BENCHMARKS dimension measured in hermetic CI via pnpm benchmarks; real-model numbers come from the local apps/measure-real runner (the CI gate runs a deterministic stub — see BENCHMARKS.md for the split). 9 surfaces (Discord, Slack, Telegram, Teams, web widget, REST, gRPC, Matrix, voice/Twilio) + 4 ticketing connectors (Zendesk, Intercom, Freshdesk, ServiceNow). 6 model adapters — Anthropic-direct, OpenAI, Gemini, Mistral, Ollama, Bedrock — all streaming. 2 vector stores + Redis/Postgres state stores. Protocol interop: MCP client + OAuth gateway, A2A v1 (server + client), AG-UI, ACP. Step-level durable execution with interrupt() HITL (@tenet/durable), agent harness (compaction + todos + subagents), WASM sandbox + wasmtime, on-prem Helm. Plus the OSS gap nobody else fills: pre-tool-call governance + approval gates + structured audit trail (@tenet/governance). Pre-1.0; APIs may change.


The honest read

This is a framework, not a model. It does not train, fine-tune, or distill a frontier LLM. Nothing here "beats" Claude / GPT / Gemini at language tasks — it calls them. The defensible claim is: TENET makes whichever frontier model you call provably more grounded, cheaper per resolution, harder to jailbreak, and the only OSS framework with built-in tool-call governance + approval gates + audit trail. That is a systems claim, and it is measurable. The BENCHMARKS gate runs every PR; numbers are produced, not asserted.

Why this beats every OSS competitor (verified June 2026)

Web-fetched competitor research surfaced the gap nobody fills:

"None evaluates a tool call against a policy before it executes, none requires approval gates by default, none ships a structured audit trail without custom integration." — Comparison of LangChain, LlamaIndex, Semantic Kernel agent frameworks (2026)

@tenet/governance ships all three: per-tool policy rules with deny/allow/require_approval, idempotent approval gate with cache, AuditSink emitting policy.allow / policy.deny / policy.require_approval / approval.decision / tool.invoked / override events. Plus we ship the things competitors miss individually:

CapabilityTENETLangChainMastraCrewAIPydantic-AIOpenAI AgentsAutoGenLlamaIndex
Pre-tool-call policy✅✗✗✗✗✗✗✗
Approval gates (HITL)✅✅ interrupt()✅ suspend/resumepartialpartial✅ (2026-04)maintenance✗
Structured audit trail✅✗✗✗✗✗✗✗
Durable execution (step-level replay)✅ @tenet/durablepartial (workflow checkpoints)✅partial✅ via Temporal✗maintenance✗
A2A protocol v1 (server + client)✅tbdtbdenterprisetbd✗maintenancetbd
AG-UI streaming frontend protocol✅✅tbd✅✅✗maintenancetbd
Agent harness (compaction · todos · subagents)✅✅ Deep Agentstbdtbdtbd✅ (2026-04)maintenancetbd
Multi-judge verifier✅✗✗✗✗✗✗✗
Deterministic claim pre-checks (numeric fabrication · quote grounding)✅✗✗✗✗✗✗✗
Claim repair loop + best-of-N verifier-ranked✅ @tenet/refine✗✗✗✗✗✗✗
Independent hallucination judge (HHEM-2.1)✅✗✗✗✗✗✗✗
BENCHMARKS gated in CI on every PR✅✗✗✗✗✗✗✗
WASM sandbox + capability tokens✅✗✗✗✗✗✗✗
Read-after-write memory consistency contract✅n/an/an/an/an/an/an/a
Streaming (token + structured)✅partial✅partial✅✅✗✗
Workflow DAG✅✅✅✅✅✅✅partial
Multi-surface (10) one config✅✗partial✗✗✗✗✗
2025-2026 high-severity CVEs03tbdtbdtbdtbdmaintenance modetbd
On-prem Helm with security defaults✅✗✗✗✗✗✗✗
Pricingfree Apache-2.0freefreefreefreefreefreefree

The hermetic CI gate measures the framework's measurement plane (verifier contract, guardrail patterns, retriever scoring, gate logic) on bundled fixtures with a deterministic stub SUT. Real frontier-model numbers come from @tenet/app-measure-real which the operator runs locally with their own API key against tenet-public-slice-0.1.0 or any swap-in dataset. The split is deliberate and the BENCHMARKS doc spells it out.

What you can build

  • Community moderation + Q&A bots for Telegram / Discord servers — verified answers with citations, no hallucinated URLs.
  • Customer-support agents that integrate with Zendesk / Intercom / Freshdesk / ServiceNow, emit outcome events for SLA reporting, and respect platform rate limits as a first-class scheduling primitive.
  • Internal IT helpdesk agents for Slack / Teams with episodic memory and HITL escalation.
  • Developer/API support agents with code-aware RAG and MCP-native tool use, tool execution sandboxed in WASM with capability tokens.
  • Web chat widgets (SSE + JWT session) served from a single REST/gRPC backend.
  • Multi-channel hybrid deployments — same agent reachable on TG, Slack, web, and a ticketing webhook from one tenet.config.ts.

BENCHMARKS — measured, not marketed

Measured on the hermetic stub gate (pnpm benchmarks) on 2026-06-04, scorer @tenet/eval-metrics@0.1.0. Every PR re-runs the gate.

DimensionTargetMeasured (stub)nVerdict
Hallucination rate (claim contract)< 0.0010.0000020PASS
Groundedness rate (atomic claims)> 0.991.0000020PASS
Cost per resolution (USD)< 0.05 stub / < 0.01 real0.00820PASS
TTFT p95 (ms)< 250 stub / < 200 real206.35020PASS
E2E latency p95 (ms)< 1500641.00020PASS
Tool-call success rate> 0.991.0000012PASS
Prompt-injection block rate> 0.991.0000020PASS
Privacy / PII block rate> 0.991.000008PASS
Retrieval recall @ K=10> 0.9951.0000010PASS
Router decision accuracy> 0.951.0000010PASS
Determinism (byte-stable verifier)= 1.01.0000010PASS

Reproduce:

pnpm install
pnpm --filter @tenet/eval-metrics build
pnpm --filter @tenet/eval-measure build
pnpm benchmarks            # table + exit 1 on FAIL/MISSING
pnpm benchmarks:json       # JSON report for CI artifacts

Real-model numbers (operator runs locally with their API key):

ANTHROPIC_API_KEY=sk-ant-... \
  node --experimental-strip-types apps/measure-real/examples/anthropic-sonnet.ts

Method-of-attack per dimension + the honest stub-vs-real target split: docs/BENCHMARKS.md.

Quick comparison

TENETLangChain / LangGraphMastraCrewAIIntercom FinDecagon / Sierra
LicenseApache-2.0MITMITMITproprietaryproprietary
Verification by default✅ multi-judge + 13 starter Principlesopt-inopt-inopt-inopaqueopaque
Source-grounded URL allow-list✅ enforced at decodemanualmanualmanualyesyes
Hallucination judge (independent)✅ HHEM-2.1 adapter shipsDIYDIYDIYopaqueopaque
Multi-surface, one config✅ 10 shippedDIYpartialDIYtheir UI onlytheir UI only
OpenTelemetry GenAI semconv✅ nativepartialpartialpartialnono
Outcome events (resolved / handed-off / disqualified / qualified)✅ first-classDIYDIYDIYyes (closed)yes (closed)
Prompt-injection + system-prompt-leak guards✅ ship by defaultopt-inopt-inopt-inopaqueopaque
WASM-sandboxed tool execution✅ shipped (reference + wasmtime adapter)nononon/an/a
MCP-native tools + OAuth gateway✅ shippedpartialpartialnonono
BENCHMARKS rows gated in CI✅ every PRnononon/a (closed)n/a (closed)
2025-2026 high-severity CVEs03tbdtbdn/an/a
Self-host / on-prem✅ Helm chart ships✅✅✅noenterprise add-on
TypeScript-native✅partial✅python-firstn/an/a
Pricingfree (Apache-2.0)freefreefree$0.99 / resolutionenterprise quote

12 differentiators

  1. Verification by default. Atomic-claim multi-judge with strict + permissive merge, ClaimPreFilter / JudgePromptDecorator strategies, default SourcePicker + 12 trap-pattern adversarial examples. Not opt-in.
  2. Independent hallucination judge. @tenet/judge-hhem ships the Vectara HHEM-2.1 contract; @tenet/judge-hhem-onnx wraps any local transformers.js / onnxruntime pipeline. No self-judging on the headline row.
  3. Source-grounded only. URLs constrained to a compiled allow-list at decode time; URL-allowlist filter ships built-in.
  4. CVE-class mistakes prevented by construction. No pickle-style deserialization. Parameterized queries. Regex-validated filter keys. Path-allowlist on every file I/O via PathHandle + openPath(). Secret<T> opaque type refuses JSON.stringify. SBOM + Trivy + audit in CI.
  5. WASM-sandboxed tools with capability tokens. Tools declare networkAllowList / readPaths / envKeys; the gateway checks subset against tenant policy; the runtime enforces fuel + memory + wall-clock. Reference Node 22 runtime + production @tenet/tools-wasm-sandbox-wasmtime adapter. Per-call fresh instance.
  6. MCP-native tools with OAuth gateway. RFC 9207 iss validation per the 2026-07-28 spec RC; per-tenant policy with allowedTools wildcard, trustedIssuers allow-list, per-tool rate-limit; audit sink emits allow/deny/error per call.
  7. All surfaces, one config. Discord · Slack · Telegram · MS Teams · web widget (SSE + JWT) · REST (OpenAPI 3.1) · gRPC (.proto + handlers) · Matrix · voice (Twilio Media Streams) — 9 surfaces — plus 4 ticketing connectors (Zendesk · Intercom · Freshdesk · ServiceNow), one composition.
  8. Rate-limit as scheduling primitive. Per-platform token buckets + circuit breakers (Telegram 1/s/chat, Discord 10k-invalid IP-ban floor, Slack 4-tier, Teams 50 RPS/tenant, Zendesk 200–2500 rpm + endpoint sub-caps).
  9. Outcome events as first-class telemetry. resolved / handed_off / disqualified / qualified emitted per conversation; OutcomeEmitter has re-entrancy guard + swallowed-error reporter.
  10. Hybrid contextual retrieval. BM25 + dense + RRF fusion + Anthropic-style contextual chunk prepending; pluggable reranker.
  11. BENCHMARKS gated in hermetic CI. Every PR runs pnpm benchmarks. Failing dimension or missing measurement blocks merge. Measured.json artifacted with 90-day retention for trend tracking.
  12. Multi-tenancy + on-prem from day 1. Per-tenant indexes, per-tenant secret stores, OTel export to customer-controlled collector. Helm chart with runAsNonRoot UID 10001, readOnlyRootFilesystem, NetworkPolicy, HPA.

Tech stack

  • TypeScript strict + exactOptionalPropertyTypes + noUncheckedIndexedAccess, target ES2024
  • Node 22 LTS (uses Promise.withResolvers, Object.groupBy, Array.toSorted, AbortSignal.any, native WebAssembly)
  • pnpm workspaces monorepo (31 workspace projects)
  • OpenTelemetry GenAI semantic conventions native
  • Models: Anthropic-direct · AWS Bedrock · wrappers, never lock-in
  • Vector stores: pgvector (≤50M vectors) · Qdrant (multi-tenant, 100M+)
  • State: Redis adapter
  • Memory adapters: mem0 · Letta · Zep
  • Hallucination judge: Vectara HHEM-2.1 (adapter + ONNX wrapper)
  • WASM runtimes: Node-22 reference + wasmtime adapter

What ships today (24 workspace packages, 611 tests)

AreaPackages
Core@tenet/core · @tenet/verifier · @tenet/policy · @tenet/telemetry · @tenet/governance
Pipeline@tenet/rate-limit · @tenet/retrieval · @tenet/router · @tenet/memory · @tenet/guardrails · @tenet/memory-adapters
Execution@tenet/durable (step-level replay + interrupt() HITL) · @tenet/workflow · @tenet/harness (compaction · todos · subagents) · @tenet/refine (boundary gate · repair loop · best-of-N · self-consistency)
Interop@tenet/a2a (A2A v1 server + client) · @tenet/ag-ui · @tenet/acp · @tenet/streaming · @tenet/tool-call-repair · @tenet/net-policy
Surfaces@tenet/surface-telegram · @tenet/surface-discord · @tenet/surface-slack · @tenet/surface-teams · @tenet/surface-web-widget · @tenet/surface-rest · @tenet/surface-grpc · @tenet/surface-matrix · @tenet/surface-twilio
Connectors@tenet/connectors-ticketing (Zendesk · Intercom · Freshdesk · ServiceNow)
Models@tenet/models-anthropic · @tenet/models-openai · @tenet/models-google · @tenet/models-mistral · @tenet/models-ollama · @tenet/models-bedrock — all streaming
Stores@tenet/stores-vector-pgvector · @tenet/stores-vector-qdrant · @tenet/stores-state-redis · @tenet/stores-state-postgres
Tools@tenet/tools-mcp · @tenet/tools-mcp-gateway · @tenet/tools-wasm-sandbox · @tenet/tools-wasm-sandbox-wasmtime
Eval@tenet/eval-harness · @tenet/eval-metrics · @tenet/eval-measure
Judges@tenet/judge-hhem · @tenet/judge-hhem-onnx
Apps@tenet/app-quickstart (pnpm quickstart) · @tenet/app-community-bot · @tenet/app-enterprise-support · @tenet/app-measure-real
Infrainfra/helm/tenet (on-prem Helm chart with security defaults)

Repo layout

packages/    core · verifier · policy · telemetry · governance · rate-limit · retrieval ·
             router · memory · guardrails · memory-adapters · durable · workflow · harness ·
             a2a · ag-ui · acp · streaming · tool-call-repair · net-policy · skills · voice ·
             voice-openai-realtime · distillation · eval-mining · rerank-cohere · tools-mcp ·
             tools-mcp-gateway · tools-wasm-sandbox · tools-wasm-sandbox-wasmtime ·
             judge-hhem · judge-hhem-onnx · telemetry-otlp
surfaces/    telegram · discord · slack · teams · web-widget · rest · grpc · matrix · twilio
connectors/  ticketing (4 vendors)
models/      anthropic · openai · google · mistral · ollama · bedrock
stores/      vector/{pgvector,qdrant} · state/{redis,postgres}
apps/        community-bot · enterprise-support · measure-real
eval/        harness · metrics · measure
infra/       helm/tenet
docs/        ARCHITECTURE · BENCHMARKS · ROADMAP · CHANGELOG · PENDING · RESEARCH-PICKS · RESEARCH-PASS-2

Quick start

Your first agent in 5 minutes — zero API keys required:

git clone https://github.com/zakky8/TENET.git
cd TENET
pnpm install
pnpm quickstart             # REPL: retrieve → draft → verify → cited answer
you > what is the verifier?
agent > The TENET verifier extracts atomic claims from a draft, then runs a
        strict judge and a permissive judge over each claim...
        sources: docs/ARCHITECTURE.md#verifier
        [verified=true abstained=false 5ms]

The quickstart runs the FULL pipeline (claim extraction + strict/permissive judging) against a deterministic offline stub. Set ANTHROPIC_API_KEY and the identical composition runs against a real model. Then:

pnpm typecheck              # builds all packages in topological order
pnpm test                   # 1036 tests across 83 suites
pnpm benchmarks             # hermetic BENCHMARKS gate (exits 1 on FAIL)

Documentation

DocWhat it covers
docs/ARCHITECTURE.mdComponent map, security stance, RAG reference stack, MVP slice, risks
docs/BENCHMARKS.mdMeasured column per dimension + stub-vs-real target split + reproduce commands
docs/ROADMAP.mdPhases 0..4 status, anti-goals
docs/CHANGELOG.mdPer-release notes
docs/PENDING.mdOpen research questions + suggested execution order
docs/RESEARCH-PICKS.mdTier-1 defaults (eval layer, OTel version, judge model)
docs/RESEARCH-PASS-2.mdClosed-tool architecture, vector-store benchmark at 100M, MCP adoption, reranker head-to-head
CONTRIBUTING.mdSetup, branching, PR rules, project values
SECURITY.mdThreat model + hardened-by-construction guarantees + disclosure policy

Status

PhaseScopeStatus
0 — ScaffoldRepo, CI, core types, verifier, policy, telemetry, docs✅ done
1 — MVPretrieval · rate-limit · guardrails · router · memory · telegram · bedrock · pgvector · redis · community-bot · eval-harness✅ done
2 — Surfaces + adaptersDiscord, Slack, web widget, Anthropic-direct, Qdrant, mem0/Letta/Zep, MCP-native✅ done
3 — EnterpriseTeams, REST, gRPC, 4 ticketing connectors, enterprise-support, Helm, MCP gateway, WASM sandbox✅ done
4 — Beat the benchmarkseval-metrics scorers · eval-measure runner + CI gate · HHEM judge adapter + ONNX wrapper · wasmtime runtime adapter · real-model measure runner · BENCHMARKS measured column✅ done

What's still pending

Honest list. None of these block shipping; each is a real-model leverage rung.

  1. Real HHEM-2.1 model bytes — @tenet/judge-hhem-onnx wraps any HF / ONNX pipeline; the operator brings the model weights (license / install choice is theirs).
  2. Real-fetch integration job in CI — secret-gated workflow that runs apps/measure-real/examples/anthropic-sonnet.ts when ANTHROPIC_API_KEY is set. Never blocks PRs from forks.
  3. Self-distillation loop — BENCHMARKS Dim 2 ("cost < $0.01") depends on weekly LoRA fine-tunes of a cheap-path model on verified flagship traces. Trace capture ships; the fine-tune pipeline does not.
  4. Auto-grown eval set — Dim 7 (">1M assertions") needs the assertion-mining + dedup-via-embedding-clustering pipeline. Not shipped.
  5. PENDING.md Bucket A + D research refresh — Anthropic SDK feature surface (D1), MCP spec changelog (D2), reranker head-to-head (D6), closed-tool architecture (A1), vector-store cost-perf at 1B (A2).

Contributing

PRs welcome. Start with CONTRIBUTING.md. The project values (verification by default · source-grounded only · hardened by construction · measured not marketed) are non-negotiable — contributions that conflict get pushback even if the code is correct.

License

Apache-2.0. Use freely in commercial and non-commercial projects. The Apache patent grant matters for an agent framework that intersects multiple SaaS platform APIs.

Acknowledgements

  • Anthropic — Constitutional AI (Bai et al. 2022 → our STANDARD_PRINCIPLES), Citations API, Contextual Retrieval recipe, Claude Agent SDK.
  • DeepMind / OpenAI prompt-engineering research — Chain-of-Verification (Dhuliawala et al. 2023), Reflexion (Shinn et al. 2023), Sycophancy (Sharma et al. 2024 → our P11).
  • Wang et al. 2024 — Speculative RAG (-51% latency / +13% accuracy → our SpeculativeAgent).
  • Cormack et al. 2009 — Reciprocal Rank Fusion → our reciprocalRankFusion.
  • The OpenTelemetry GenAI semantic conventions working group.
  • The Model Context Protocol spec maintainers + the 2026-07-28 RC iss-validation contributors.
  • Vectara (HHEM-2.1) + Patronus Lynx — independent hallucination judges.
  • Bytecode Alliance (wasmtime) — production WASM engine reference.
  • Mastra, Pydantic-AI, Letta, OpenAI Agents SDK — OSS agent-framework ecosystem we compete with and learn from.

Keywords for discovery: open source AI agent framework · TypeScript agent SDK · LangChain alternative · hallucination-resistant LLM agent · RAG with citations · multi-channel chatbot framework · enterprise customer support agent · IT helpdesk AI · Telegram bot AI · Discord bot AI · Slack bot AI · Microsoft Teams AI · AWS Bedrock agent · Anthropic Claude agent · self-hosted agent · on-prem AI · OpenTelemetry GenAI · Model Context Protocol (MCP) · MCP OAuth gateway · A2A protocol · Agent2Agent · AG-UI · durable agent execution · human-in-the-loop interrupt · agent harness · WASM tool sandbox · capability tokens · Vectara HHEM-2.1 · source-grounded LLM · verification-first agent · adversarial-reviewed agent framework · BENCHMARKS-gated CI · OSS agent monorepo

Ver o original no GitHub

Bom para

Relacionados