Document Chunk Converter
Junhaozhang-127/document-chunk-converter
Converte documentos >32MB entre MD, PDF e DOCX via chunking inteligente
Ver no GitHubSkill para Claude Code que processa arquivos grandes (>32MB) dividindo-os em chunks, convertendo cada pedaço entre formatos MD/PDF/DOCX e remontando o resultado final. Resolve limitações de leitura de arquivos grandes e conversão de formatos de forma transparente e automática.
Instalação
npx convez-ai Junhaozhang-127/document-chunk-converter1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.
Métricas
Casos de uso
- Converter PDFs grandes para DOCX mantendo formatação
- Processar relatórios volumosos que excedem limite de 32MB
- Automatizar pipeline de conversão de formato para documentos em lote
Veredito

Solução pragmática para contornar limitações técnicas reais do Claude Code com arquivos grandes. Arquitetura modular e bem documentada, mas requer configuração específica por SO (especialmente Windows) e depende de ferramentas externas (pandoc, python-docx, weasyprint).
README
A tradução para o português deste README ainda está na fila.
Document Chunk Converter
English | 中文
A Claude Code Skill that transparently converts large document files (>32MB) between MD, PDF, and DOCX formats by slicing them into manageable chunks, converting each chunk, and reassembling the result.
Problem
Claude Code cannot read files larger than ~32MB. When format conversion is needed (e.g., PDF to DOCX), tools like pandoc or python-docx also struggle with large files. This Skill solves both problems: it automatically chunks large files below the read limit, converts each piece, and reassembles the output — all without user intervention.
Architecture
large-file.pdf (>32MB)
│
▼
┌─────────────────────────────────────────────────┐
│ 1. chunker.py │
│ • Detects file type (MD/PDF/DOCX) │
│ • Splits at natural boundaries where possible│
│ • Falls back to fixed-size chunking if needed│
│ • Output: chunks/ + manifest.json │
└──────────────────────┬──────────────────────────┘
│
▼
┌─────────────────────────────────────────────────┐
│ 2. converter.py │
│ • Reads manifest.json │
│ • Converts each chunk to target format │
│ • Marks failed chunks in manifest │
│ • Output: converted/ + manifest.json │
└──────────────────────┬──────────────────────────┘
│
▼
┌─────────────────────────────────────────────────┐
│ 3. assembler.py │
│ • Merges all successful chunks │
│ • Skips failed chunks with warnings │
│ • Auto-cleans intermediate files │
│ • Output: output.docx (or .md/.pdf) │
└─────────────────────────────────────────────────┘
Installation
pip install -r requirements.txt
Windows-specific dependencies
For MD→PDF and DOCX→PDF conversions on Windows, install one of:
| Strategy | Package | Notes |
|---|---|---|
| Word COM (recommended) | pywin32 | Requires Microsoft Word installed |
| WeasyPrint | weasyprint (included) | Requires GTK3 runtime |
| pdfkit (fallback) | pdfkit | Requires wkhtmltopdf |
Usage
Three-step workflow
# Step 1: Chunk the large file into ~4MB pieces
python scripts/chunker.py large-input.pdf -o /tmp/lfc_chunks --max-mb 4
# Step 2: Convert each chunk to the target format
python scripts/converter.py /tmp/lfc_chunks/manifest.json docx -o /tmp/lfc_converted
# Step 3: Assemble converted chunks into final output
python scripts/assembler.py /tmp/lfc_converted/manifest.json -o output.docx
One-liner
python scripts/chunker.py big.pdf -o /tmp/lfc_chunks && \
python scripts/converter.py /tmp/lfc_chunks/manifest.json docx -o /tmp/lfc_converted && \
python scripts/assembler.py /tmp/lfc_converted/manifest.json -o output.docx
CLI Reference
chunker.py
| Flag | Description | Default |
|---|---|---|
-o, --output-dir | Output directory for chunks (required) | — |
-m, --max-mb | Max chunk size in MB | 4.0 |
--manifest | Manifest filename | manifest.json |
converter.py
| Flag | Description |
|---|---|
-o, --output-dir | Output directory for converted chunks (required) |
target_format | One of: md, pdf, docx |
assembler.py
| Flag | Description |
|---|---|
-o, --output | Output file path (required) |
--keep-chunks | Preserve intermediate chunk files (default: auto-cleanup) |
Supported Conversions
| Source | Target | Method |
|---|---|---|
| MD | DOCX | markdown → HTML → python-docx |
| MD | Word COM / weasyprint / pdfkit (3-strategy fallback) | |
| DOCX | MD | python-docx text extraction (headings + tables preserved) |
| DOCX | Word COM, fallback: DOCX → MD → PDF | |
| MD | pdfplumber text extraction | |
| DOCX | PDF → MD → DOCX (two-step) |
Note: Two-step conversions (PDF→DOCX, DOCX→PDF fallback) use MD as an intermediate format. Some formatting may be simplified compared to direct conversion.
Chunking Strategy
The chunker uses a hybrid strategy to split files intelligently:
Level 1: Natural boundaries (preferred)
- MD files: Splits at heading boundaries (
#,##,###, etc.), keeping sections together - PDF files: Groups pages, estimating average page size to stay under the max chunk size
- DOCX files: Splits at body element boundaries (paragraphs and tables), preserving document structure
Level 2: Fixed-size fallback
When a single natural segment (e.g., one very long section) exceeds the max chunk size, that segment is further split into fixed-size pieces. This ensures no chunk exceeds the limit while preserving structure wherever possible.
Each chunk's boundary_type (natural or fixed) is recorded in the manifest. During assembly, natural boundaries get proper separators (page breaks in DOCX, --- in MD).
Manifest Format
The manifest.json file is the data contract between pipeline stages:
{
"source": "large-report.pdf",
"source_type": "pdf",
"total_chunks": 14,
"chunks_dir": "/tmp/lfc_chunks",
"target_format": "docx",
"chunks": [
{
"index": 0,
"file": "chunk_0000.pdf",
"size_bytes": 3987456,
"boundary_type": "natural",
"page_range": [1, 12],
"converted_file": "/tmp/lfc_converted/chunk_0000.docx",
"status": "ok"
}
]
}
Fields added by each stage:
- chunker:
source,source_type,total_chunks,chunks_dir,chunks[] - converter:
target_format,chunks[].converted_file,chunks[].status - assembler: reads manifest, produces final output
Output Reporting
After a successful conversion, you'll see:
Assembly complete: output.docx
Chunks: 14 merged, 0 skipped (of 14)
If some chunks failed to convert:
Assembly complete with warnings: output.docx
Chunks: 12 merged, 2 skipped (of 14)
Warning: 2 chunks could not be converted. Content may be incomplete.
Dependencies
Core (required for all conversions)
| Package | Version | Purpose |
|---|---|---|
python-docx | ≥0.8.11 | DOCX read/write, MD→DOCX |
PyPDF2 | ≥3.0.0 | PDF chunking, PDF assembly |
pdfplumber | ≥0.10.0 | PDF text extraction |
markdown | ≥3.5.0 | MD→HTML rendering |
beautifulsoup4 | ≥4.12.0 | HTML parsing for MD→DOCX |
weasyprint | ≥60.0 | MD→PDF (fallback strategy) |
Optional
| Package | Purpose |
|---|---|
markdown-it-py | Alternative MD→HTML renderer |
pdfkit | Alternative MD→PDF (requires wkhtmltopdf) |
pywin32 | Windows Word COM automation (fastest PDF output) |
Fault Tolerance
- Failed conversions are non-fatal: The assembler skips chunks marked
"status": "failed"and merges only successful ones - Partial output: If 2 of 14 chunks fail, you still get 12 chunks' worth of content in the output file
- Failed chunks preserved: Use
--keep-chunksto inspect failed chunks for debugging - Automatic cleanup: Intermediate files are removed after successful assembly (disable with
--keep-chunks)
Limitations
- Not for search/analysis: This Skill is for format conversion only. To search or edit large files, use
grep,ripgrep, or split manually - Format fidelity: Two-step conversions (PDF→MD→DOCX, DOCX→MD→PDF) simplify formatting. Images, complex tables, and layout may not be perfectly preserved
- Text-only PDF extraction: pdfplumber extracts text; images, charts, and vector graphics in PDFs are lost
- No binary formats: Images, audio, video, and other binary formats are not supported
File Structure
document-chunk-converter/
├── SKILL.md # Claude Code Skill definition
├── README.md # English documentation
├── README_zh.md # Chinese documentation
├── requirements.txt # Python dependencies
├── .gitignore # Ignores __pycache__, .tmp, etc.
└── scripts/
├── chunker.py # Step 1: Smart file slicing
├── converter.py # Step 2: Per-chunk format conversion
└── assembler.py # Step 3: Merge + cleanup
Common Issues
"Unsupported file type"
The chunker only accepts .md, .pdf, and .docx files. Check the file extension.
Missing dependency errors
If you see ImportError for a specific conversion, install the relevant optional package. The error message will tell you which one.
Chunks failing to convert
A single chunk may fail if it contains corrupt data or unsupported formatting. The assembler will skip it — inspect the chunk with --keep-chunks if needed.
Verification
To verify the Skill is correctly installed after setup, run:
python -m py_compile scripts/chunker.py scripts/converter.py scripts/assembler.py
All three should report no errors.