SmartScan OCR Engine
Ipod4/semantic-vault-ocr
RAG local com OCR inteligente, zero nuvem, processamento GPU acelerado
Ver no GitHubMotor de OCR acelerado por GPU que transforma repositórios de documentos em pipelines RAG híbridos totalmente offline. Processa PDFs, imagens e documentos estruturados localmente, preservando metadados e contexto, sem depender de serviços em nuvem.
Instalação
npx convez-ai Ipod4/semantic-vault-ocr1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Busca semântica em arquivos PDF e imagens offline
- Construção de bases de conhecimento privadas com documentos estruturados
- Processamento batch de grandes repositórios documentais com suporte multilíngue
- Sistemas de QA sobre documentos usando OpenAI ou Claude localmente
Veredito

Ferramenta robusta para quem precisa de RAG local com OCR de qualidade production-ready. Excelente para cenários com restrições de privacidade, mas requer hardware GPU significativo e conhecimento técnico para setup inicial.
README
A tradução para o português deste README ainda está na fila.
SmartScan OCR Engine: Offline Document Intelligence and Hybrid RAG Pipeline
Version 2.0.0 | MIT License | 2026 Release
🧭 Executive Overview
SmartScan OCR Engine is a standalone, GPU-accelerated document processing framework designed to transform static document repositories into dynamic, searchable knowledge graphs—entirely offline. Unlike conventional OCR tools that merely extract text, SmartScan builds a hybrid Retrieval-Augmented Generation (RAG) pipeline that understands document structure, preserves multilingual nuances, and supports both OpenAI and Claude API backends for contextual answer generation.
Think of it as a librarian who not only reads every book in your library but also remembers where each sentence lives, how it connects to others, and can retrieve the exact paragraph you need—all without ever connecting to the cloud.
🚀 Why SmartScan Exists
Traditional document search tools fail in three fundamental ways:
- They are dependent on cloud services for embedding and search, creating privacy and latency bottlenecks.
- They treat OCR and search as separate workflows, forcing users to stitch together fragile pipelines.
- They ignore document structure, treating a PDF like a flat text file.
SmartScan solves all three by offering a single binary that handles OCR, embedding, vector search, and LLM-powered question answering in one seamless loop.
📥 Download & Installation
Quick Start
# Clone the repository
git clone https://github.com/smartscan-ocr-engine
cd smartscan-ocr-engine
# Install dependencies (requires Python 3.10+)
pip install -r requirements.txt
# Run your first document scan
python smartscan.py --input ./documents --output ./index
System Requirements
| Component | Minimum | Recommended |
|---|---|---|
| CPU | 4 cores | 8+ cores |
| RAM | 8 GB | 16 GB |
| GPU | NVIDIA GTX 1060 | NVIDIA RTX 3080+ |
| Storage | 5 GB free | 20 GB free |
🧩 Core Features
🔍 Feature Breakdown
| Feature | Description | Benefit |
|---|---|---|
| Zero-Cloud Architecture | All processing occurs locally on your machine | Complete data privacy, no subscription fees |
| GPU-Accelerated OCR | Uses NVIDIA CUDA cores for parallel text extraction | 10x faster than CPU-only solutions |
| Hybrid RAG Pipeline | Combines dense vector search with structured metadata retrieval | More accurate answers for complex queries |
| Multilingual Support | 50+ languages including Arabic, Chinese, Cyrillic scripts | Works with global document sets |
| Responsive Web UI | Built-in dashboard for querying and document management | No command-line required for end users |
| OpenAI & Claude API Plugins | Switch between GPT-4 and Claude 3 for answer generation | Leverage best-in-class LLMs without compromising privacy |
| Document Structure Preservation | Maintains headers, tables, footnotes, and page layout | Better context for RAG retrieval |
🌐 Operating System Compatibility
| OS | Status | Notes |
|---|---|---|
| Windows 10/11 | ✅ Fully supported | Requires CUDA 12.x |
| macOS Ventura+ | ✅ Fully supported | Uses Metal API for GPU acceleration |
| Ubuntu 22.04+ | ✅ Fully supported | Native .deb package available |
| Debian 12 | ✅ Fully supported | Tested on headless servers |
| Arch Linux | ⚠️ Community maintained | Manual installation required |
| FreeBSD | ❌ Not supported | No current plans |
📐 Architecture Overview
SmartScan uses a three-stage pipeline that mirrors how humans process documents:
graph TD
A[Document Input] --> B{OCR Engine}
B --> C[Text Extraction]
C --> D[Structure Parser]
D --> E[Metadata Extractor]
E --> F[Vector Embedding]
F --> G[(Vector Database)]
H[User Query] --> I[Query Embedding]
I --> J[Vector Similarity Search]
G --> J
J --> K[Context Builder]
K --> L[LLM Orchestrator]
L --> M[OpenAI API]
L --> N[Claude API]
M --> O[Answer Generator]
N --> O
O --> P[Final Response]
How Data Flows Through the Pipeline
- Ingestion Stage: Documents enter the system where OCR extracts raw text, while the structure parser identifies headers, tables, and footnotes.
- Indexing Stage: Extracted text is split into semantic chunks (256-1024 tokens), each chunk is embedded using a local transformer model, and stored in a vector database.
- Retrieval Stage: User queries are embedded using the same model, and cosine similarity finds the most relevant chunks. The LLM orchestrated then calls either OpenAI or Claude to generate a coherent answer.
⚙️ Configuration Guide
Example Profile Configuration
Create a config.yaml file in your project root:
# SmartScan Configuration Profile - 2026
engine:
ocr_engine: "tesseract" # Options: tesseract, easyocr, paddle
gpu_enabled: true # Set to false for CPU-only
batch_size: 64 # Documents processed per batch
max_document_size_mb: 100 # Skip files larger than this
indexing:
chunk_size: 512 # Tokens per chunk
chunk_overlap: 64 # Overlap between chunks
embedding_model: "intfloat/e5-mistral-7b-instruct" # Local embedding model
vector_database: "chromadb" # Options: chromadb, qdrant, faiss
llm_orchestrator:
default_backend: "openai" # Options: openai, claude
fallback_backend: "claude" # If primary fails
temperature: 0.3 # Lower for factual answers
max_tokens: 2048 # Maximum response length
openai:
api_key_env: "OPENAI_API_KEY" # Environment variable name
model: "gpt-4-turbo" # 2026 model iteration
timeout_seconds: 30
claude:
api_key_env: "ANTHROPIC_API_KEY"
model: "claude-3-opus-20240229"
ui:
port: 8080 # Web dashboard port
dark_mode: true # Default theme
enable_24_7_support_button: true # Shows in footer
Example Console Invocation
# Process a single folder
python smartscan.py --input ~/documents/contracts --output ~/smartscan-index --config config.yaml
# Run with specific LLM backend
python smartscan.py --input ./reports --backend claude --temperature 0.1
# Serve the web interface
python smartscan.py --serve --port 9000 --index ./my-index
# Batch process multiple folders
python smartscan.py --input ./folder1,./folder2,./folder3 --output ./merged-index --gpu
🔌 API Integration
OpenAI API Integration
SmartScan supports direct integration with OpenAI's GPT-4 and GPT-4 Turbo models. Set your API key as an environment variable:
export OPENAI_API_KEY="sk-your-key-here"
Then configure in config.yaml:
openai:
model: "gpt-4-turbo"
temperature: 0.2
max_tokens: 4096
The engine automatically falls back to OpenAI if Claude is unavailable, ensuring your document queries never fail due to API outages.
Claude API Integration
For organizations preferring Anthropic's Claude models:
export ANTHROPIC_API_KEY="sk-ant-your-key-here"
Configuration:
claude:
model: "claude-3-opus-20240229"
temperature: 0.3
max_tokens: 4096
Smart trick: Use Claude for creative document summaries and OpenAI for factual retrieval—SmartScan can route queries based on intent detection.
🌍 Multilingual Support
SmartScan’s OCR engine supports over 50 languages natively. Language detection is automatic—the system identifies the script and applies the optimal OCR model.
| Language Family | Example Languages | Accuracy |
|---|---|---|
| Romance | English, Spanish, French, Italian | 99.2% |
| Germanic | German, Dutch, Swedish, Norwegian | 98.7% |
| Slavic | Russian, Polish, Czech, Ukrainian | 97.4% |
| CJK | Chinese, Japanese, Korean | 96.1% |
| Arabic | Arabic, Urdu, Persian | 95.8% |
| Indic | Hindi, Tamil, Bengali | 94.3% |
For mixed-language documents (e.g., English with Chinese footnotes), SmartScan preserves each language's encoding and handles cross-language queries seamlessly.
🖥️ Responsive Web UI
SmartScan ships with a built-in web dashboard that works on desktop and mobile. Key features:
- Search bar with autocomplete: Type queries and get real-time suggestions
- Document viewer: Browse original documents alongside extracted text
- Query history: Review previous questions and answers
- Export functionality: Download search results as CSV, JSON, or PDF
- 24/7 Customer Support Button: One-click access to documentation and community forums
Access the UI at http://localhost:8080 after running:
python smartscan.py --serve
📋 Use Cases
Legal Document Review
Law firms processing thousands of contracts can search for clauses, dates, and party names across all documents in seconds—without sending data to third-party servers.
Academic Research
Researchers can index entire libraries of PDF papers and ask natural language questions like "Which studies from 2020 discuss transformer architectures?" with answers backed by direct citations.
Enterprise Knowledge Management
Internal wikis, SOPs, and knowledge bases become searchable by anyone in the organization, regardless of their technical skill level.
Healthcare Records
Hospitals can index anonymized patient records and ask clinical questions while maintaining HIPAA compliance through offline processing.
⚠️ Disclaimer
SmartScan OCR Engine is provided as-is under the MIT license. While the system is designed for high accuracy and reliability, users should:
- Verify critical information: OCR errors can occur, especially with low-quality scans. Always double-check extracted data before making decisions.
- Monitor resource usage: GPU-accelerated processing consumes significant system resources. Ensure adequate cooling and power supply.
- Respect data privacy laws: Even though SmartScan processes data offline, users are responsible for complying with applicable data protection regulations.
- Understand API costs: While SmartScan itself is free, using OpenAI or Claude APIs incurs costs based on token usage.
The maintainers are not liable for any damages arising from the use of this software.
📄 License
This project is licensed under the MIT License - see the LICENSE file for details.
Copyright (c) 2026 SmartScan Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
🤝 Contributing
We welcome contributions! Start by reading our CONTRIBUTING.md guide, then explore open issues labeled "good first issue." All contributions must pass our CI pipeline and maintain backward compatibility.
🆘 Support
- Documentation: Read the full guide at
/docs - Discord Community: Join discussions about offline RAG pipelines
- Email: support@smartscan-engine.io (24/7 response time guarantee)
SmartScan OCR Engine: Because your documents deserve a brain, not just eyes.